-- Logs begin at Mon 2022-07-25 18:23:50 UTC, end at Mon 2022-07-25 20:21:40 UTC. -- Jul 25 19:06:23.788264 dv-cinder-850939-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 25 19:06:23.802430 dv-cinder-850939-1-iscsi devstack@keystone.service[80229]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 25 19:06:23.805625 dv-cinder-850939-1-iscsi devstack@keystone.service[80229]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jul 25 19:06:23 2022] *** Jul 25 19:06:23.805625 dv-cinder-850939-1-iscsi devstack@keystone.service[80229]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 25 19:06:23.805625 dv-cinder-850939-1-iscsi devstack@keystone.service[80229]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 25 19:06:23.805625 dv-cinder-850939-1-iscsi devstack@keystone.service[80229]: nodename: dv-cinder-850939-1-iscsi Jul 25 19:06:23.805625 dv-cinder-850939-1-iscsi devstack@keystone.service[80229]: machine: x86_64 Jul 25 19:06:23.805625 dv-cinder-850939-1-iscsi devstack@keystone.service[80229]: clock source: unix Jul 25 19:06:23.805625 dv-cinder-850939-1-iscsi devstack@keystone.service[80229]: pcre jit disabled Jul 25 19:06:23.805625 dv-cinder-850939-1-iscsi devstack@keystone.service[80229]: detected number of CPU cores: 4 Jul 25 19:06:23.805625 dv-cinder-850939-1-iscsi devstack@keystone.service[80229]: current working directory: / Jul 25 19:06:23.805625 dv-cinder-850939-1-iscsi devstack@keystone.service[80229]: detected binary path: /usr/bin/uwsgi-core Jul 25 19:06:23.806316 dv-cinder-850939-1-iscsi devstack@keystone.service[80229]: your processes number limit is 47680 Jul 25 19:06:23.806316 dv-cinder-850939-1-iscsi devstack@keystone.service[80229]: your memory page size is 4096 bytes Jul 25 19:06:23.806316 dv-cinder-850939-1-iscsi devstack@keystone.service[80229]: detected max file descriptor number: 2048 Jul 25 19:06:23.806316 dv-cinder-850939-1-iscsi devstack@keystone.service[80229]: lock engine: pthread robust mutexes Jul 25 19:06:23.806316 dv-cinder-850939-1-iscsi devstack@keystone.service[80229]: thunder lock: enabled Jul 25 19:06:23.806316 dv-cinder-850939-1-iscsi devstack@keystone.service[80229]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 25 19:06:23.806316 dv-cinder-850939-1-iscsi devstack@keystone.service[80229]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 25 19:06:23.884378 dv-cinder-850939-1-iscsi devstack@keystone.service[80229]: Python main interpreter initialized at 0x5611721539c0 Jul 25 19:06:23.884378 dv-cinder-850939-1-iscsi devstack@keystone.service[80229]: python threads support enabled Jul 25 19:06:23.884378 dv-cinder-850939-1-iscsi devstack@keystone.service[80229]: your server socket listen backlog is limited to 100 connections Jul 25 19:06:23.884378 dv-cinder-850939-1-iscsi devstack@keystone.service[80229]: your mercy for graceful operations on workers is 90 seconds Jul 25 19:06:23.884948 dv-cinder-850939-1-iscsi devstack@keystone.service[80229]: mapped 403077 bytes (393 KB) for 2 cores Jul 25 19:06:23.885056 dv-cinder-850939-1-iscsi devstack@keystone.service[80229]: *** Operational MODE: preforking *** Jul 25 19:06:23.885625 dv-cinder-850939-1-iscsi devstack@keystone.service[80229]: *** uWSGI is running in multiple interpreter mode *** Jul 25 19:06:23.885625 dv-cinder-850939-1-iscsi devstack@keystone.service[80229]: spawned uWSGI master process (pid: 80229) Jul 25 19:06:23.885914 dv-cinder-850939-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 25 19:06:23.886881 dv-cinder-850939-1-iscsi devstack@keystone.service[80229]: spawned uWSGI worker 1 (pid: 80237, cores: 1) Jul 25 19:06:23.887395 dv-cinder-850939-1-iscsi devstack@keystone.service[80229]: spawned uWSGI worker 2 (pid: 80238, cores: 1) Jul 25 19:06:23.890686 dv-cinder-850939-1-iscsi devstack@keystone.service[80229]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 25 19:06:30.826678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 19:06:30.826678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: from cryptography.utils import int_from_bytes Jul 25 19:06:30.827972 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 19:06:30.827972 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: from cryptography.utils import int_from_bytes Jul 25 19:06:30.830483 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 19:06:30.830483 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: from cryptography.utils import int_from_bytes Jul 25 19:06:30.831325 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 19:06:30.831325 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: from cryptography.utils import int_from_bytes Jul 25 19:06:30.883100 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 25 19:06:30.883390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 25 19:06:30.883635 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] command line args: [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 25 19:06:30.884044 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 25 19:06:30.884364 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ================================================================================ {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 25 19:06:30.884747 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] admin_token = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.885010 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] config_dir = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.885204 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.885392 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] config_source = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.885582 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] debug = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.885790 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.886179 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.886345 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.886598 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.886854 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.887092 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.887271 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] log_config_append = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.887433 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.887638 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] log_dir = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.887832 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] log_file = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.888101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.888482 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.888745 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.889011 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.889190 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.889353 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.889608 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.889917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.890093 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.890317 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.890733 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.890926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.891096 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.891294 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.892433 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.892738 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 25 19:06:30.892979 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.893184 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 25 19:06:30.893389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] publish_errors = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.893582 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] command line args: [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 25 19:06:30.893777 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.893972 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 25 19:06:30.894154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.894154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.894526 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ================================================================================ {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 25 19:06:30.894729 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.894729 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.895081 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] admin_token = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.895268 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] standard_threads = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.895268 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.895674 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] config_dir = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.895868 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.896062 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.896245 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.896245 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] use_journal = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.896656 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] config_source = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.896832 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] use_json = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.896832 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] use_stderr = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.897180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] debug = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.897378 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] use_syslog = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.897561 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.897800 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.897800 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.898161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.898354 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.898553 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.898749 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.898749 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.899102 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.899295 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.899545 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.899737 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.899737 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.900099 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] list_limit = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.900291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.900476 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] log_config_append = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.900668 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.900668 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.901052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.901245 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.901420 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] log_dir = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.901622 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.901883 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] log_file = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.902057 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.902057 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.902391 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.902588 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.902588 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.902961 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.903154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.903349 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.903557 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.903557 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.903922 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.904123 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.904310 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.904504 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.904702 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.904889 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.905074 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.905273 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.905273 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.905630 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.905818 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.905818 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.906189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.906370 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.906370 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.906703 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.906884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.906884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.907278 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.908029 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.908211 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.908415 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.908415 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.908782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.908980 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.909175 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.909369 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.909369 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.909734 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.909984 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.909984 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.909984 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.909984 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.910665 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.910862 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.911175 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] publish_errors = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.911362 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.911569 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.911770 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.911957 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.912157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.912351 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.912535 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.912720 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.912896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.912896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.913260 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.913443 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.913635 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] standard_threads = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.913828 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.913828 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.914184 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.914383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.914383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.914728 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.914910 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.915106 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.915289 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.915289 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.915666 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] use_journal = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.915846 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.916048 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] use_json = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.916233 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.916233 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.916608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] use_stderr = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.916804 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.916990 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] use_syslog = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.917171 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.917171 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.917516 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 19:06:30.917721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.917721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.918076 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.918274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.918458 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.918649 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.918649 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.919119 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.919303 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.919303 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.919695 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.919880 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.919880 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.920263 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.920470 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.920470 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.920829 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.921007 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.921007 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.921345 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.921523 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.921523 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.921866 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.922049 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.922049 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.922409 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.922594 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.922779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.922959 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.922959 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.923334 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.923623 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.923623 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.923981 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.924195 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.924195 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.924518 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.924690 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.924855 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.925043 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.925043 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.925921 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.926126 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.926126 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.926456 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.926633 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.926824 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.927010 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.927010 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.927366 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.927569 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.927569 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.927939 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.928144 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.928144 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.928512 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.928717 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.928903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.929090 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.929274 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.929464 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.929664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.929861 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.929861 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.930211 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.930405 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.930405 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.930755 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.930947 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.external = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.930947 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.931308 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.931491 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.931707 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.931917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.931917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.password = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.932267 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.932462 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.token = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.932669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.932868 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.932868 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.933240 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.933428 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.933428 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.933775 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.933960 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.934161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.934355 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.934531 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.934721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.934721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.935085 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.935271 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.935271 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.935648 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.935836 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.936023 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.936220 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.936220 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.936569 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.936756 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.936756 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.937102 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.937303 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.937303 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.937660 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.937848 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.938030 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.938217 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.938415 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.938602 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.938602 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.938927 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.939113 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.939318 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.939656 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.939656 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.940051 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.940236 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.940236 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.940593 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.940786 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.940786 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.941150 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.941337 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.941337 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.941698 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.941895 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.942097 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.942294 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.942294 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.942647 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.942831 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.942831 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.943191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.943375 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.943375 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.943830 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.944015 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.944015 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.944387 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.944576 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.944760 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.944946 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.944946 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.945295 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.945502 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.945502 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.945852 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.946047 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.946234 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.946417 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.946618 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.946848 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.947056 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.947254 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.947254 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.948159 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.948342 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.948528 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.948743 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.948743 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.949109 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.949302 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.949486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.949671 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.949671 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.950016 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.950203 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.950203 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.950570 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.950758 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.950758 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.951135 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.951321 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.951321 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.951698 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.951886 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.952090 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.952279 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.952279 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.952656 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.952844 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.952844 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.953220 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.953407 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.953407 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.953753 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.953978 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.954179 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.954384 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.954384 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.954735 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.954918 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.954918 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.955281 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.955482 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.955482 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.955936 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.956148 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.956336 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.956525 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.956525 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.956887 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.957087 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.957087 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.957436 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.957634 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.957634 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.958005 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.958224 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.958224 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.958573 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.958758 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.958758 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.959124 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.959310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.959310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.959680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.959887 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.960095 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.960283 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.960283 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.960600 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.960787 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.960787 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.961277 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.961478 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.961478 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.961846 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.962032 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.962032 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.962397 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.962583 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.962769 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.962955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.963144 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.963326 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.963326 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.963734 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.963922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.964109 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.964289 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.964488 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.964687 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.964874 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.965061 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.965061 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.965409 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.965597 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.965597 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.965960 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.966159 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.966159 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.966513 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.966956 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.967154 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.967343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.967343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.967723 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.967911 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.967911 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.968278 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.968467 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.968467 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.968820 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.969011 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.969011 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.969372 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.969560 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.969560 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.970585 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.970774 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.970774 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.971154 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.971633 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.971633 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.972010 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.972196 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.972382 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.972580 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.972580 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.972931 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] auth.external = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.973116 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.973116 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.973462 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.973663 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.973663 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.974029 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.974217 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.974217 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] role.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.974581 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.974770 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] role.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.974770 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.975117 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] auth.password = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.975427 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.975427 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.975839 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] auth.token = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.976033 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.976218 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.976415 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.976415 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.976778 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.976980 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.976980 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.977387 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.977573 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.977573 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.977920 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.978121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.978121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.978469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.978655 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.978655 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.979000 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.979211 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.979399 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.979687 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.979687 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.980044 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] credential.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.980232 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.980232 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.980643 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.980829 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.980829 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.981179 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.981367 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.981565 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.981752 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.981752 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.982100 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.982290 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.982290 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.982653 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.982841 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.982841 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.983202 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.983389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.983597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.983806 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.983994 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.984179 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.984179 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.984531 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.984731 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.984731 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.985083 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.986231 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.986231 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.986806 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.987010 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.987010 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.987367 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.987598 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.987598 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.987952 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.988155 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.988155 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.988502 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.988690 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.988690 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.989038 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.989238 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.989437 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.989628 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.989628 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.989979 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.990165 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.990165 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.990530 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.990764 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.990764 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.991102 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.991274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.connection = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.991274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.991798 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.992004 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.992194 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.992383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.992383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.992734 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.992920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.992920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.993284 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.993473 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.993473 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.993824 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.994009 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.994009 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.994378 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.994567 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.994567 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.994919 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.995108 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.995108 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.995482 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.995699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.995699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.996052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.996237 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.996237 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.996713 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.996900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 25 19:06:30.997089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.997089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.997089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.997089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.997089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.997089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.997089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.997089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.997089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.998608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.998608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.998608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.998608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.998608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.998608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.998608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.998608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:30.998608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.000156 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.000156 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.000156 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.000156 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.000156 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.000156 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.000156 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.000156 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.000156 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.002209 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.002209 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.002209 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.002209 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.002209 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.002209 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.002209 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.002209 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.002209 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.003893 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.003893 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.003893 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.003893 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.003893 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.003893 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.003893 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.003893 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.003893 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.005418 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.005418 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.005418 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.005418 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.005418 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.005418 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.005418 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.006589 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.006589 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.006589 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.007095 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.007344 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.007731 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.008093 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.008456 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.008821 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.009174 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.009561 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.010098 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.010470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.010845 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.011219 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.011629 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.012017 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.012380 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.012743 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.013104 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.013466 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.013838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.014218 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.014583 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.014962 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.015314 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.015732 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.016095 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.016500 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.017417 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.017616 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.017935 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.018183 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.018402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.018596 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.018791 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.019109 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.019473 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.019901 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.020302 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.020674 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.021048 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.021423 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.021781 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.022151 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.022518 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.022878 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.023238 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] role.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.023659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] role.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.024013 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.024384 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.024770 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.025144 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.025517 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.025887 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.026265 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.026637 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.027022 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.027402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.027681 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.027885 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.028249 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.028625 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.029009 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.029567 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.029926 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.030295 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.030667 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.031047 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.031417 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.031829 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.032246 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.032618 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.032980 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.033427 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.033876 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.034232 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.034607 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.034986 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.035353 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.035763 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.036122 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.036549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.036934 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] token.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.037447 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.037844 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.038364 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.038741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.039113 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.039477 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.039888 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.040261 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.040644 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.041018 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.041393 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.041776 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.042164 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.042538 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.042895 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.043263 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.043742 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.044128 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.connection = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.044509 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.044888 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.045249 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.045619 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.045976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.046345 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.046704 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.047065 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.047433 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.048295 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.048716 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.049392 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.049585 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.049981 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.050253 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.050509 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.050886 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.051153 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 19:06:31.051337 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 25 19:06:31.060544 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.060774 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.080623 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.080842 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.081066 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.090694 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.114065 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.114344 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.121864 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.164045 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.182108 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.182355 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.205603 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.206160 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.255358 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.255632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.255828 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.256053 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.256276 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.256547 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.256908 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.257147 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.257377 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.257607 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.257831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.258053 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.282560 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.282815 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.283228 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.283634 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.283785 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.283899 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.284005 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.284128 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.284244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.284345 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.284444 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.284745 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.285202 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.285430 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.297643 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.299477 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.312884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.313202 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.313588 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.313944 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.314340 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.314533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.314731 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.314976 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.315214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.315602 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.316002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.316363 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.316740 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.317096 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 19:06:31.335372 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.335595 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.335811 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 19:06:31.335989 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 19:06:31.336195 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.336313 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.336501 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 19:06:31.336783 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.336885 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 19:06:31.337044 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.337308 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.337438 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.337564 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.337684 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.338081 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.338446 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.351743 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.352237 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.352711 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.353195 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.354283 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.354553 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.354830 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.355117 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.363826 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 19:06:31.365705 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 19:06:31.368359 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 19:06:31.368929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.369348 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 19:06:31.369605 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.369765 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.369912 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.370203 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.370543 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.370653 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.370827 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.371103 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.371363 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.371464 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.371773 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.372029 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.372148 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.372355 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.372686 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.373107 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.373606 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.374045 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.374460 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.411980 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.414239 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 19:06:31.414913 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.415228 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.417697 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 19:06:31.417987 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.421542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 19:06:31.422197 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.422496 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.424360 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 19:06:31.424832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.425123 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.432104 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 19:06:31.432634 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.433955 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 19:06:31.434257 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.438068 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 19:06:31.439240 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.439928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 19:06:31.440197 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.440384 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.440515 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.440781 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.441058 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.441220 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.441353 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.441632 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.441907 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.442007 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.442405 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.442833 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.443383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.463658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.464236 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.465032 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.465333 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.469478 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 19:06:31.471846 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 19:06:31.474028 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.474469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.475832 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.476879 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.480195 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 19:06:31.481712 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 19:06:31.484492 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 19:06:31.486159 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 19:06:31.489298 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.489626 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.489917 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.490304 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.490503 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.490671 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.490824 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.490979 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.491415 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.491942 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.492099 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.492307 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.512013 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 19:06:31.512316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.512586 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.512850 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.514345 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 19:06:31.514858 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.515273 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.515705 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.522185 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.524100 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.524384 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.526540 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.527861 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 19:06:31.528158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.528453 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.528750 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.529036 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.529634 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.530084 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.535102 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 19:06:31.535689 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.535979 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.536412 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.536702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.544669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 19:06:31.544963 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.545444 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.545773 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.546213 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.546552 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.546845 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.553792 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 19:06:31.554321 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.554683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.555129 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.555580 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.555894 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.556188 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.569666 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 19:06:31.573981 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 19:06:31.576887 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 19:06:31.577974 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.580943 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.581643 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 19:06:31.583131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 19:06:31.583410 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.583962 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.585532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.587348 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.589354 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 19:06:31.589842 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.590288 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.593049 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.595610 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 19:06:31.599411 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.599751 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.600166 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.600452 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.600761 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.602677 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 19:06:31.606258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.606557 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.606971 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.607268 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.610487 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 19:06:31.610928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.611222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.617468 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 19:06:31.617767 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.618059 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.619532 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 19:06:31.619841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.620130 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.620439 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.620747 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.621142 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.621424 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.621700 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.622006 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.622291 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.622568 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.622834 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.623211 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.623476 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.626597 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 19:06:31.627164 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.627430 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.627738 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.628029 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.628441 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.628726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.629053 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.629316 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.629629 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.629884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.630153 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.630440 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.630708 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 19:06:31.665144 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: WSGI app 0 (mountpoint='') ready in 8 seconds on interpreter 0x5611721539c0 pid: 80238 (default app) Jul 25 19:06:31.668943 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ef991-cc79-4905-8f3b-d557ef2f1278 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:31.669239 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ef991-cc79-4905-8f3b-d557ef2f1278 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:31.669523 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ef991-cc79-4905-8f3b-d557ef2f1278 None None] PATH_INFO: `/v3` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:31.671404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: WSGI app 0 (mountpoint='') ready in 8 seconds on interpreter 0x5611721539c0 pid: 80237 (default app) Jul 25 19:06:31.674672 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1/1] 10.222.0.18 () {50 vars in 791 bytes} [Mon Jul 25 19:06:31 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:06:41.312810 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98db0b4-dd1c-4120-b055-aa9837ba8af6 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:41.314896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98db0b4-dd1c-4120-b055-aa9837ba8af6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:41.314896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98db0b4-dd1c-4120-b055-aa9837ba8af6 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:41.318812 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1/2] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:06:41 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:06:41.324954 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210bf80-b9e9-478c-9db5-1f1604c1e2da None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:41.325556 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210bf80-b9e9-478c-9db5-1f1604c1e2da None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:41.325556 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210bf80-b9e9-478c-9db5-1f1604c1e2da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:41.474306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: INFO dbcounter [None req-7210bf80-b9e9-478c-9db5-1f1604c1e2da None None] Registered counter for database keystone Jul 25 19:06:41.513581 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG oslo_db.sqlalchemy.engines [None req-7210bf80-b9e9-478c-9db5-1f1604c1e2da None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80238) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 25 19:06:41.522037 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writer thread running {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 25 19:06:41.733973 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG passlib.handlers.bcrypt [None req-7210bf80-b9e9-478c-9db5-1f1604c1e2da None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80238) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 25 19:06:41.734396 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG passlib.handlers.bcrypt [None req-7210bf80-b9e9-478c-9db5-1f1604c1e2da None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80238) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 25 19:06:41.799554 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7210bf80-b9e9-478c-9db5-1f1604c1e2da None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:41.801464 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7210bf80-b9e9-478c-9db5-1f1604c1e2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:42.057010 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 2/3] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:06:41 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 733 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 25 19:06:42.363575 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98db0b4-dd1c-4120-b055-aa9837ba8af6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:42.533956 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: INFO dbcounter [None req-b98db0b4-dd1c-4120-b055-aa9837ba8af6 None None] Registered counter for database keystone Jul 25 19:06:42.573103 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG oslo_db.sqlalchemy.engines [None req-b98db0b4-dd1c-4120-b055-aa9837ba8af6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80237) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 25 19:06:42.584856 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writer thread running {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 25 19:06:42.849420 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98db0b4-dd1c-4120-b055-aa9837ba8af6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:42.857525 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33caf85f-46ad-41bc-b91c-7e7020599e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:42.858767 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33caf85f-46ad-41bc-b91c-7e7020599e53 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:42.859274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33caf85f-46ad-41bc-b91c-7e7020599e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:42.859862 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33caf85f-46ad-41bc-b91c-7e7020599e53 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:42.884749 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33caf85f-46ad-41bc-b91c-7e7020599e53 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:42.961947 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: WARNING keystone.common.rbac_enforcer.enforcer [None req-33caf85f-46ad-41bc-b91c-7e7020599e53 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 25 19:06:42.962323 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33caf85f-46ad-41bc-b91c-7e7020599e53 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:42.977582 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 2/4] 10.222.0.18 () {58 vars in 1324 bytes} [Mon Jul 25 19:06:42 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 19:06:42.988011 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210bf80-b9e9-478c-9db5-1f1604c1e2da None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:43.018646 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210bf80-b9e9-478c-9db5-1f1604c1e2da None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:43.027113 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a407e1-1c25-401e-a675-2f849ad101c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:43.029313 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a407e1-1c25-401e-a675-2f849ad101c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:43.029766 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a407e1-1c25-401e-a675-2f849ad101c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:43.030260 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a407e1-1c25-401e-a675-2f849ad101c8 admin admin] PATH_INFO: `/v3/projects/9565eeec7ac7457abc5ee445dbf84c18` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:43.033466 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a407e1-1c25-401e-a675-2f849ad101c8 admin admin] RBAC: Authorizing `identity:get_project(project_id=9565eeec7ac7457abc5ee445dbf84c18)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:43.148746 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: WARNING keystone.common.rbac_enforcer.enforcer [None req-49a407e1-1c25-401e-a675-2f849ad101c8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 25 19:06:43.149235 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a407e1-1c25-401e-a675-2f849ad101c8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:43.158304 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 3/5] 10.222.0.18 () {56 vars in 1207 bytes} [Mon Jul 25 19:06:42 2022] GET /identity/v3/projects/9565eeec7ac7457abc5ee445dbf84c18 => generated 344 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:06:45.935661 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb765a04-b80a-4f7c-bd29-c0d25bd05a35 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:45.937487 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb765a04-b80a-4f7c-bd29-c0d25bd05a35 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:45.937487 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb765a04-b80a-4f7c-bd29-c0d25bd05a35 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:45.945498 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 3/6] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:06:45 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:06:45.955940 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b32067-7d2c-4a18-8a4d-cf779dd1586c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:45.956747 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b32067-7d2c-4a18-8a4d-cf779dd1586c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:45.956920 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b32067-7d2c-4a18-8a4d-cf779dd1586c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:45.993258 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-50b32067-7d2c-4a18-8a4d-cf779dd1586c None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:45.996254 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-50b32067-7d2c-4a18-8a4d-cf779dd1586c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:46.009446 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 4/7] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:06:45 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 25 19:06:46.049545 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb765a04-b80a-4f7c-bd29-c0d25bd05a35 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:46.069261 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb765a04-b80a-4f7c-bd29-c0d25bd05a35 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:46.074365 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf2248-12f1-418a-8860-824260ad3e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:46.075044 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf2248-12f1-418a-8860-824260ad3e74 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:46.075362 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf2248-12f1-418a-8860-824260ad3e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:46.075672 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf2248-12f1-418a-8860-824260ad3e74 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:46.086439 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf2248-12f1-418a-8860-824260ad3e74 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:46.088552 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf2248-12f1-418a-8860-824260ad3e74 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:46.101159 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 4/8] 10.222.0.18 () {58 vars in 1324 bytes} [Mon Jul 25 19:06:46 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 25 19:06:46.107147 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b32067-7d2c-4a18-8a4d-cf779dd1586c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:46.117897 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b32067-7d2c-4a18-8a4d-cf779dd1586c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:46.122619 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b5dce8-3b66-4bd8-8f4d-7889064e5336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:46.123660 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5dce8-3b66-4bd8-8f4d-7889064e5336 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:46.123953 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5dce8-3b66-4bd8-8f4d-7889064e5336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:46.124236 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5dce8-3b66-4bd8-8f4d-7889064e5336 admin admin] PATH_INFO: `/v3/users/35d700f1abf943789efbf8a08bd9b587` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:46.126312 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b5dce8-3b66-4bd8-8f4d-7889064e5336 admin admin] RBAC: Authorizing `identity:get_user(user_id=35d700f1abf943789efbf8a08bd9b587)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:46.128320 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b5dce8-3b66-4bd8-8f4d-7889064e5336 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:46.130040 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 5/9] 10.222.0.18 () {56 vars in 1198 bytes} [Mon Jul 25 19:06:46 2022] GET /identity/v3/users/35d700f1abf943789efbf8a08bd9b587 => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:06:48.707673 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615b4c1-498b-4ed8-8c1c-41aa21788a3b None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:48.708285 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615b4c1-498b-4ed8-8c1c-41aa21788a3b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:48.708422 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615b4c1-498b-4ed8-8c1c-41aa21788a3b None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:48.710305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 5/10] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:06:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:06:48.717765 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de64f25-800f-4d4a-8bcc-858a40d0a2c3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:48.718318 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de64f25-800f-4d4a-8bcc-858a40d0a2c3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:48.718787 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de64f25-800f-4d4a-8bcc-858a40d0a2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:48.751481 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4de64f25-800f-4d4a-8bcc-858a40d0a2c3 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:48.753681 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4de64f25-800f-4d4a-8bcc-858a40d0a2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:48.930898 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 6/11] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:06:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:06:48.934940 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e36855c-face-4ddd-9d71-f51c34e70ecb None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:48.935233 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e36855c-face-4ddd-9d71-f51c34e70ecb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:48.935689 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e36855c-face-4ddd-9d71-f51c34e70ecb None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:48.937156 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 6/12] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:06:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:06:48.947664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35f852-3a50-4bf3-ae2b-46dc6cca3fe7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:48.948168 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35f852-3a50-4bf3-ae2b-46dc6cca3fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:48.948679 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35f852-3a50-4bf3-ae2b-46dc6cca3fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:48.974495 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e36855c-face-4ddd-9d71-f51c34e70ecb None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:48.981944 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3a35f852-3a50-4bf3-ae2b-46dc6cca3fe7 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:48.983432 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3a35f852-3a50-4bf3-ae2b-46dc6cca3fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:49.021474 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:49.117941 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 7/13] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:06:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:06:49.142130 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35f852-3a50-4bf3-ae2b-46dc6cca3fe7 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:49.209764 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e36855c-face-4ddd-9d71-f51c34e70ecb None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:49.267597 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6679c32-4f83-46a9-b26b-cb5d60e70628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:49.268131 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6679c32-4f83-46a9-b26b-cb5d60e70628 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:49.268532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6679c32-4f83-46a9-b26b-cb5d60e70628 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:49.268902 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6679c32-4f83-46a9-b26b-cb5d60e70628 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:49.282417 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6679c32-4f83-46a9-b26b-cb5d60e70628 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:06:49.294173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-f6679c32-4f83-46a9-b26b-cb5d60e70628 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:06:49.294173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:06:49.294173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:06:49.294173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:06:49.294173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:06:49.294173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:06:49.294173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:06:49.294173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:06:49.294173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:06:49.294173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:06:49.294173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:06:49.294173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:06:49.294173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:06:49.294173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:06:49.294173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:06:49.294173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:06:49.295361 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:06:49.295361 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:06:49.295361 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:06:49.295361 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:06:49.295361 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:06:49.295361 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:06:49.295361 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:06:49.295361 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:06:49.295361 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:06:49.295361 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:06:49.295361 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:06:49.295361 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:06:49.295361 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:06:49.295361 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:06:49.295361 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:06:49.295361 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:06:49.296532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:06:49.296532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:06:49.296532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:06:49.296532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:06:49.296532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:06:49.296532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:06:49.296532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:06:49.296532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:06:49.296532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:06:49.296532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:06:49.296532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:06:49.296532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:06:49.296532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 7/14] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Jul 25 19:06:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:06:49.302418 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6679c32-4f83-46a9-b26b-cb5d60e70628 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:49.306092 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35f852-3a50-4bf3-ae2b-46dc6cca3fe7 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:49.350653 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:49.354011 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b712f5-92d3-454b-8dd5-f34c254bf283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:49.354690 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b712f5-92d3-454b-8dd5-f34c254bf283 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:49.354978 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b712f5-92d3-454b-8dd5-f34c254bf283 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:49.355248 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b712f5-92d3-454b-8dd5-f34c254bf283 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:49.387602 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:49.456926 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b712f5-92d3-454b-8dd5-f34c254bf283 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:49.458440 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b712f5-92d3-454b-8dd5-f34c254bf283 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:49.480923 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6679c32-4f83-46a9-b26b-cb5d60e70628 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:49.531014 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf242c-8b81-46b3-8f18-9d496fbe3ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:49.533154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf242c-8b81-46b3-8f18-9d496fbe3ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:49.534118 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf242c-8b81-46b3-8f18-9d496fbe3ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:49.534700 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf242c-8b81-46b3-8f18-9d496fbe3ac6 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:49.535881 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaf242c-8b81-46b3-8f18-9d496fbe3ac6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:06:49.536451 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaf242c-8b81-46b3-8f18-9d496fbe3ac6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:49.539448 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaf242c-8b81-46b3-8f18-9d496fbe3ac6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:49.569040 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 8/15] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:06:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:06:49.570973 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf242c-8b81-46b3-8f18-9d496fbe3ac6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:49.591418 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 8/16] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:06:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 19:06:49.598560 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b712f5-92d3-454b-8dd5-f34c254bf283 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:49.689081 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:49.708820 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:49.717715 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf242c-8b81-46b3-8f18-9d496fbe3ac6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:49.764033 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b712f5-92d3-454b-8dd5-f34c254bf283 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:49.766591 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1748ba1f-a322-46fd-a024-bde8471e453b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:49.767541 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748ba1f-a322-46fd-a024-bde8471e453b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:49.767832 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748ba1f-a322-46fd-a024-bde8471e453b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:49.768109 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748ba1f-a322-46fd-a024-bde8471e453b None admin] PATH_INFO: `/v3/users/35d700f1abf943789efbf8a08bd9b587` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:49.769805 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1748ba1f-a322-46fd-a024-bde8471e453b None admin] RBAC: Authorizing `identity:get_user(user_id=35d700f1abf943789efbf8a08bd9b587)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:49.771290 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1748ba1f-a322-46fd-a024-bde8471e453b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:49.773182 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 9/17] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:06:49 2022] GET /identity/v3/users/35d700f1abf943789efbf8a08bd9b587 => generated 247 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:06:49.779067 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1748ba1f-a322-46fd-a024-bde8471e453b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:49.803146 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7db90a-b59a-421d-b3ac-459728ef08a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:49.803907 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7db90a-b59a-421d-b3ac-459728ef08a9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:49.804197 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7db90a-b59a-421d-b3ac-459728ef08a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:49.804477 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7db90a-b59a-421d-b3ac-459728ef08a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:49.815901 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7db90a-b59a-421d-b3ac-459728ef08a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:49.817409 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7db90a-b59a-421d-b3ac-459728ef08a9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:49.839241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-1c7db90a-b59a-421d-b3ac-459728ef08a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:06:49.839241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:06:49.839241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:06:49.839241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:06:49.839241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:06:49.839241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:06:49.839241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:06:49.839241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:06:49.839241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:06:49.839241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:06:49.839241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:06:49.839241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:06:49.839241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:06:49.839241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:06:49.839241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:06:49.841144 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:06:49.841144 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:06:49.841144 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:06:49.841144 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:06:49.841144 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:06:49.841144 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:06:49.841144 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:06:49.841144 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:06:49.841144 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:06:49.841144 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:06:49.841144 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:06:49.841144 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:06:49.841144 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:06:49.841144 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:06:49.841144 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:06:49.841144 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:06:49.841144 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:06:49.843019 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:06:49.843019 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:06:49.843019 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:06:49.843019 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:06:49.843019 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 9/18] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:06:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:06:49.848138 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7db90a-b59a-421d-b3ac-459728ef08a9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:49.922655 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1748ba1f-a322-46fd-a024-bde8471e453b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:49.965251 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcf9794-e1f4-4f20-b84d-46730131683d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:49.966088 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf9794-e1f4-4f20-b84d-46730131683d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:49.966372 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf9794-e1f4-4f20-b84d-46730131683d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:49.966911 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf9794-e1f4-4f20-b84d-46730131683d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:49.968466 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcf9794-e1f4-4f20-b84d-46730131683d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:49.970268 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcf9794-e1f4-4f20-b84d-46730131683d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:49.972239 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 10/19] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:06:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:06:49.975727 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7db90a-b59a-421d-b3ac-459728ef08a9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:49.978552 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcf9794-e1f4-4f20-b84d-46730131683d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:49.979051 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:50.018690 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:50.026960 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96038eb4-888f-48f5-94d7-6665bbe50709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:50.028207 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96038eb4-888f-48f5-94d7-6665bbe50709 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:50.028691 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96038eb4-888f-48f5-94d7-6665bbe50709 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:50.029207 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96038eb4-888f-48f5-94d7-6665bbe50709 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:50.030608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96038eb4-888f-48f5-94d7-6665bbe50709 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:06:50.031200 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96038eb4-888f-48f5-94d7-6665bbe50709 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:50.034763 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96038eb4-888f-48f5-94d7-6665bbe50709 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:50.060209 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 10/20] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:06:50.113491 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcf9794-e1f4-4f20-b84d-46730131683d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:50.160360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d8a8c-2014-468c-b619-490c1eeb1c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:50.161226 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d8a8c-2014-468c-b619-490c1eeb1c08 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:50.161598 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d8a8c-2014-468c-b619-490c1eeb1c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:50.161967 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d8a8c-2014-468c-b619-490c1eeb1c08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:50.162959 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d8a8c-2014-468c-b619-490c1eeb1c08 None admin] RBAC: Adding query filter params (role.id=ed7fec18968241efa76a0ee769d96da9, scope.domain.id=default, user.id=35d700f1abf943789efbf8a08bd9b587) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:06:50.163681 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d8a8c-2014-468c-b619-490c1eeb1c08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:50.165917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d8a8c-2014-468c-b619-490c1eeb1c08 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:50.180281 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 11/21] 10.222.0.18 () {56 vars in 1322 bytes} [Mon Jul 25 19:06:49 2022] GET /identity/v3/role_assignments?user.id=35d700f1abf943789efbf8a08bd9b587&scope.domain.id=default&role.id=ed7fec18968241efa76a0ee769d96da9 => generated 233 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:06:53.899774 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c66a8a7-43f2-40d5-86c8-34385fd26101 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:53.900556 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c66a8a7-43f2-40d5-86c8-34385fd26101 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:53.900935 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c66a8a7-43f2-40d5-86c8-34385fd26101 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:53.903789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 11/22] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:06:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:06:53.913571 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64910562-d59a-4124-94fc-9d28a25a4073 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:53.914342 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64910562-d59a-4124-94fc-9d28a25a4073 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:53.914342 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64910562-d59a-4124-94fc-9d28a25a4073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:53.942609 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG passlib.handlers.bcrypt [None req-64910562-d59a-4124-94fc-9d28a25a4073 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80237) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 25 19:06:53.943099 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG passlib.handlers.bcrypt [None req-64910562-d59a-4124-94fc-9d28a25a4073 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80237) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 25 19:06:54.001143 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-64910562-d59a-4124-94fc-9d28a25a4073 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:54.004169 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-64910562-d59a-4124-94fc-9d28a25a4073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:54.202882 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 12/23] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:06:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:06:54.287901 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c66a8a7-43f2-40d5-86c8-34385fd26101 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:54.505912 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a041e8-7286-4d7e-906f-e3def9c2091d None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:54.506403 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a041e8-7286-4d7e-906f-e3def9c2091d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:54.506890 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a041e8-7286-4d7e-906f-e3def9c2091d None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:54.510668 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 13/24] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:06:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:06:54.519723 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566a7cd-7a2e-4083-b3fd-6330b2609c30 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:54.520218 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566a7cd-7a2e-4083-b3fd-6330b2609c30 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:54.520710 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566a7cd-7a2e-4083-b3fd-6330b2609c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:54.571561 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c66a8a7-43f2-40d5-86c8-34385fd26101 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:54.586397 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1566a7cd-7a2e-4083-b3fd-6330b2609c30 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:54.588539 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1566a7cd-7a2e-4083-b3fd-6330b2609c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:54.664156 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:54.698102 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893783f1-5f74-4bef-9bc1-c5bfc6f46e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:54.699298 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893783f1-5f74-4bef-9bc1-c5bfc6f46e71 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:54.699624 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893783f1-5f74-4bef-9bc1-c5bfc6f46e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:54.699917 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893783f1-5f74-4bef-9bc1-c5bfc6f46e71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:54.745572 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893783f1-5f74-4bef-9bc1-c5bfc6f46e71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:54.748686 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893783f1-5f74-4bef-9bc1-c5bfc6f46e71 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:54.840914 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 14/25] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:06:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:06:54.851905 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ccb012-2a6e-40fe-b005-45d3f891d058 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:54.852684 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ccb012-2a6e-40fe-b005-45d3f891d058 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:54.853231 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ccb012-2a6e-40fe-b005-45d3f891d058 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:54.856516 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 15/26] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:06:54 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:06:54.858997 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca6f83-fcde-4a80-b5c1-19076a080961 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:54.860661 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca6f83-fcde-4a80-b5c1-19076a080961 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:54.861232 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca6f83-fcde-4a80-b5c1-19076a080961 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:54.871001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 16/27] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:06:54 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:06:54.873449 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c3395-4d9a-46df-a69a-0346372a74a0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:54.873955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c3395-4d9a-46df-a69a-0346372a74a0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:54.874470 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3c3395-4d9a-46df-a69a-0346372a74a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:54.920874 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-cc3c3395-4d9a-46df-a69a-0346372a74a0 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:54.923636 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cc3c3395-4d9a-46df-a69a-0346372a74a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:54.935805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-893783f1-5f74-4bef-9bc1-c5bfc6f46e71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:06:54.935805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:06:54.935805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:06:54.935805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:06:54.935805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:06:54.935805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:06:54.935805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:06:54.935805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:06:54.935805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:06:54.935805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:06:54.935805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:06:54.935805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:06:54.935805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:06:54.935805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:06:54.935805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:06:54.940350 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:06:54.940350 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:06:54.940350 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:06:54.940350 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:06:54.940350 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:06:54.940350 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:06:54.940350 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:06:54.940350 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:06:54.940350 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:06:54.940350 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:06:54.940350 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:06:54.940350 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:06:54.940350 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:06:54.940350 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:06:54.940350 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:06:54.940350 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:06:54.940350 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:06:54.943664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:06:54.943664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:06:54.943664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:06:54.943664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:06:54.943664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 12/28] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:06:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 653 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:06:54.943664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b45910-9e7b-41e4-b698-51ad89ba8eeb None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:54.943664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b45910-9e7b-41e4-b698-51ad89ba8eeb None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:54.943664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b45910-9e7b-41e4-b698-51ad89ba8eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:54.980103 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-87b45910-9e7b-41e4-b698-51ad89ba8eeb None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:54.982728 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-87b45910-9e7b-41e4-b698-51ad89ba8eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:55.010358 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:55.070467 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 17/29] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:06:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 19:06:55.072006 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3c3395-4d9a-46df-a69a-0346372a74a0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:55.141325 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 13/30] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:06:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:06:55.143947 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b45910-9e7b-41e4-b698-51ad89ba8eeb None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:55.265157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:55.299743 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3c3395-4d9a-46df-a69a-0346372a74a0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:55.363863 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b45910-9e7b-41e4-b698-51ad89ba8eeb None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:55.370944 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06ed31-72bf-45c4-99d9-0b499bf6af5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:55.371217 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06ed31-72bf-45c4-99d9-0b499bf6af5f None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:55.371761 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06ed31-72bf-45c4-99d9-0b499bf6af5f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:55.372256 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06ed31-72bf-45c4-99d9-0b499bf6af5f None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:55.373855 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b06ed31-72bf-45c4-99d9-0b499bf6af5f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:55.377264 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b06ed31-72bf-45c4-99d9-0b499bf6af5f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:55.449771 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d7bb0-0d7c-4c06-8a47-8459de8d118d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:55.451397 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d7bb0-0d7c-4c06-8a47-8459de8d118d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:55.452259 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d7bb0-0d7c-4c06-8a47-8459de8d118d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:55.453008 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d7bb0-0d7c-4c06-8a47-8459de8d118d None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:55.456196 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95d7bb0-0d7c-4c06-8a47-8459de8d118d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:06:55.457159 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95d7bb0-0d7c-4c06-8a47-8459de8d118d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:55.460691 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95d7bb0-0d7c-4c06-8a47-8459de8d118d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:55.489669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 14/31] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:06:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:06:55.491883 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d7bb0-0d7c-4c06-8a47-8459de8d118d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:55.511046 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:55.724629 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d7bb0-0d7c-4c06-8a47-8459de8d118d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:55.799836 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 18/32] 10.222.0.18 () {60 vars in 1131 bytes} [Mon Jul 25 19:06:55 2022] POST /identity/v3/roles => generated 225 bytes in 729 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:06:55.802980 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9917f93-6745-4292-9873-e87ab0e789bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:55.804797 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9917f93-6745-4292-9873-e87ab0e789bb None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:55.805629 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9917f93-6745-4292-9873-e87ab0e789bb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:55.806377 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9917f93-6745-4292-9873-e87ab0e789bb None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:55.808245 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9917f93-6745-4292-9873-e87ab0e789bb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:55.808452 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06ed31-72bf-45c4-99d9-0b499bf6af5f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:55.811420 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9917f93-6745-4292-9873-e87ab0e789bb None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:56.145590 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 15/33] 10.222.0.18 () {60 vars in 1131 bytes} [Mon Jul 25 19:06:55 2022] POST /identity/v3/roles => generated 219 bytes in 655 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:06:56.147425 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9917f93-6745-4292-9873-e87ab0e789bb None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:56.200702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06ed31-72bf-45c4-99d9-0b499bf6af5f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:56.318841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:56.320082 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b502cf6-5c0b-4e8b-bf10-86e8fd398bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:56.322510 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b502cf6-5c0b-4e8b-bf10-86e8fd398bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:56.323383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b502cf6-5c0b-4e8b-bf10-86e8fd398bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:56.324264 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b502cf6-5c0b-4e8b-bf10-86e8fd398bc8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:56.368033 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b502cf6-5c0b-4e8b-bf10-86e8fd398bc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:06:56.389767 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-3b502cf6-5c0b-4e8b-bf10-86e8fd398bc8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:06:56.389767 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:06:56.389767 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:06:56.389767 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:06:56.389767 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:06:56.389767 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:06:56.389767 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:06:56.389767 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:06:56.389767 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:06:56.389767 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:06:56.389767 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:06:56.389767 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:06:56.389767 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:06:56.389767 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:06:56.389767 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:06:56.389767 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:06:56.392027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:06:56.392027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:06:56.392027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:06:56.392027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:06:56.392027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:06:56.392027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:06:56.392027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:06:56.392027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:06:56.392027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:06:56.392027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:06:56.392027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:06:56.392027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:06:56.392027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:06:56.392027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:06:56.392027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:06:56.392027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:06:56.393423 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:06:56.393423 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:06:56.393423 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:06:56.393423 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:06:56.393423 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:06:56.393423 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:06:56.393423 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:06:56.393423 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:06:56.393423 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:06:56.393423 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:06:56.393423 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:06:56.393423 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:06:56.400583 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9917f93-6745-4292-9873-e87ab0e789bb None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:56.406161 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 19/34] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Jul 25 19:06:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 606 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:06:56.409949 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23631da-b9e1-43e2-88ab-534a6b27e1ef None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:56.410918 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23631da-b9e1-43e2-88ab-534a6b27e1ef None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:56.411672 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23631da-b9e1-43e2-88ab-534a6b27e1ef None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:56.414551 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 20/35] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:06:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:06:56.423720 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23631da-b9e1-43e2-88ab-534a6b27e1ef None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:56.442504 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:56.479663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d289945-3890-4d51-b52e-ecfe8efad6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:56.480849 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d289945-3890-4d51-b52e-ecfe8efad6d9 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:56.481356 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d289945-3890-4d51-b52e-ecfe8efad6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:56.481868 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d289945-3890-4d51-b52e-ecfe8efad6d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:56.483256 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d289945-3890-4d51-b52e-ecfe8efad6d9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:56.486283 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d289945-3890-4d51-b52e-ecfe8efad6d9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:56.554979 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 16/36] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Jul 25 19:06:56 2022] POST /identity/v3/projects => generated 301 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:06:56.557248 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0c783-69c4-4ac8-ac9c-b7110b272997 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:56.557823 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0c783-69c4-4ac8-ac9c-b7110b272997 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:56.558300 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0c783-69c4-4ac8-ac9c-b7110b272997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:56.602998 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-98e0c783-69c4-4ac8-ac9c-b7110b272997 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:56.605995 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-98e0c783-69c4-4ac8-ac9c-b7110b272997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:56.652075 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23631da-b9e1-43e2-88ab-534a6b27e1ef None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:56.737184 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821eb4dd-619d-4fb9-af21-c763cb532ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:56.738308 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821eb4dd-619d-4fb9-af21-c763cb532ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:56.738802 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821eb4dd-619d-4fb9-af21-c763cb532ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:56.739285 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821eb4dd-619d-4fb9-af21-c763cb532ba7 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:56.740483 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821eb4dd-619d-4fb9-af21-c763cb532ba7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:06:56.741119 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821eb4dd-619d-4fb9-af21-c763cb532ba7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:56.751176 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821eb4dd-619d-4fb9-af21-c763cb532ba7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:56.762017 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 17/37] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:06:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:06:56.764888 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af8d303-1774-4262-84cb-7e8946fa9dae None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:56.765735 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af8d303-1774-4262-84cb-7e8946fa9dae None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:56.766424 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af8d303-1774-4262-84cb-7e8946fa9dae None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:56.776311 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 21/39] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:06:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:06:56.776623 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 18/39] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:06:56 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:06:56.779131 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4cddd-1c5f-47bf-908b-0fdf7cd20e9e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:56.779844 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4cddd-1c5f-47bf-908b-0fdf7cd20e9e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:56.780428 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4cddd-1c5f-47bf-908b-0fdf7cd20e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:56.781001 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af8d303-1774-4262-84cb-7e8946fa9dae None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:56.818189 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-31c4cddd-1c5f-47bf-908b-0fdf7cd20e9e None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:06:56.821302 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-31c4cddd-1c5f-47bf-908b-0fdf7cd20e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:06:56.905300 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:56.960030 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:56.976007 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 22/40] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:06:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:06:56.976007 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c4cddd-1c5f-47bf-908b-0fdf7cd20e9e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:57.020801 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af8d303-1774-4262-84cb-7e8946fa9dae None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:57.074504 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902694e-3138-4a7c-8200-28214b108be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:57.076082 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902694e-3138-4a7c-8200-28214b108be5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:57.076379 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902694e-3138-4a7c-8200-28214b108be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:57.076665 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902694e-3138-4a7c-8200-28214b108be5 None admin] PATH_INFO: `/v3/users/35d700f1abf943789efbf8a08bd9b587` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:57.078791 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b902694e-3138-4a7c-8200-28214b108be5 None admin] RBAC: Authorizing `identity:get_user(user_id=35d700f1abf943789efbf8a08bd9b587)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:57.080588 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b902694e-3138-4a7c-8200-28214b108be5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:57.082480 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 19/41] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:06:56 2022] GET /identity/v3/users/35d700f1abf943789efbf8a08bd9b587 => generated 247 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:06:57.083867 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902694e-3138-4a7c-8200-28214b108be5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:57.166466 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c4cddd-1c5f-47bf-908b-0fdf7cd20e9e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:57.243659 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d6b00-7a4d-476b-ae22-95ef1ff45cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:57.245102 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d6b00-7a4d-476b-ae22-95ef1ff45cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:57.245599 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d6b00-7a4d-476b-ae22-95ef1ff45cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:57.246096 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d6b00-7a4d-476b-ae22-95ef1ff45cc4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:57.249022 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d6b00-7a4d-476b-ae22-95ef1ff45cc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:57.253402 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d6b00-7a4d-476b-ae22-95ef1ff45cc4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:57.255891 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 23/42] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:06:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:06:57.257022 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d6b00-7a4d-476b-ae22-95ef1ff45cc4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:57.274200 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902694e-3138-4a7c-8200-28214b108be5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:57.298691 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:57.334123 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e0072-9bff-418e-a105-33319f8d98ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:57.335314 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e0072-9bff-418e-a105-33319f8d98ad None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:57.335873 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e0072-9bff-418e-a105-33319f8d98ad None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:57.336390 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e0072-9bff-418e-a105-33319f8d98ad None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:57.337924 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e0072-9bff-418e-a105-33319f8d98ad None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:57.340702 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e0072-9bff-418e-a105-33319f8d98ad None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:57.368291 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 20/43] 10.222.0.18 () {60 vars in 1131 bytes} [Mon Jul 25 19:06:57 2022] POST /identity/v3/roles => generated 223 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:06:57.369868 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e0072-9bff-418e-a105-33319f8d98ad None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:57.372175 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:57.433529 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d6b00-7a4d-476b-ae22-95ef1ff45cc4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:57.502455 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636177bb-31f6-4574-9f76-1f0e82de52e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:57.504000 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636177bb-31f6-4574-9f76-1f0e82de52e4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:57.504440 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636177bb-31f6-4574-9f76-1f0e82de52e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:57.504803 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636177bb-31f6-4574-9f76-1f0e82de52e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:57.506676 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636177bb-31f6-4574-9f76-1f0e82de52e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:57.508309 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636177bb-31f6-4574-9f76-1f0e82de52e4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:57.516624 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 24/44] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:06:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:06:57.518927 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636177bb-31f6-4574-9f76-1f0e82de52e4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:06:57.583451 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e0072-9bff-418e-a105-33319f8d98ad None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:57.651355 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d184c-988f-4bee-9d19-8f8de235b2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:57.652497 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d184c-988f-4bee-9d19-8f8de235b2ef None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:57.652953 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d184c-988f-4bee-9d19-8f8de235b2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:57.653425 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d184c-988f-4bee-9d19-8f8de235b2ef None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:57.654741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902d184c-988f-4bee-9d19-8f8de235b2ef None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:57.658128 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902d184c-988f-4bee-9d19-8f8de235b2ef None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:57.712871 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 21/45] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Jul 25 19:06:57 2022] POST /identity/v3/projects => generated 312 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:06:57.726534 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636177bb-31f6-4574-9f76-1f0e82de52e4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:06:57.799123 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01898fc5-d38d-4bf2-99e2-1cae63b6fe64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:06:57.800702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01898fc5-d38d-4bf2-99e2-1cae63b6fe64 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:06:57.801217 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01898fc5-d38d-4bf2-99e2-1cae63b6fe64 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:06:57.801644 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01898fc5-d38d-4bf2-99e2-1cae63b6fe64 None admin] PATH_INFO: `/v3/domains/default/users/35d700f1abf943789efbf8a08bd9b587/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:06:57.806088 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01898fc5-d38d-4bf2-99e2-1cae63b6fe64 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=35d700f1abf943789efbf8a08bd9b587, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:06:57.808894 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01898fc5-d38d-4bf2-99e2-1cae63b6fe64 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:06:57.814228 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:06:57.839246 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 25/46] 10.222.0.18 () {58 vars in 1361 bytes} [Mon Jul 25 19:06:57 2022] PUT /identity/v3/domains/default/users/35d700f1abf943789efbf8a08bd9b587/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:07:00.408662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d8e3d-f2cf-480f-b65d-e159b580e64d None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:00.409626 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d8e3d-f2cf-480f-b65d-e159b580e64d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:00.409626 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d8e3d-f2cf-480f-b65d-e159b580e64d None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:00.412602 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 22/47] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:07:00.422283 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f3ff3-2427-4520-bf5d-871b13665cec None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:00.423121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f3ff3-2427-4520-bf5d-871b13665cec None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:00.423284 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f3ff3-2427-4520-bf5d-871b13665cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:00.456425 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-488f3ff3-2427-4520-bf5d-871b13665cec None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:00.458575 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-488f3ff3-2427-4520-bf5d-871b13665cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:00.617543 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 26/48] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:07:00.660709 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d8e3d-f2cf-480f-b65d-e159b580e64d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:00.739052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:00.924162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d8e3d-f2cf-480f-b65d-e159b580e64d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:00.998315 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5791eb-ba24-4717-8649-519b2e30023d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:00.999710 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5791eb-ba24-4717-8649-519b2e30023d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:01.000851 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5791eb-ba24-4717-8649-519b2e30023d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:01.001353 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5791eb-ba24-4717-8649-519b2e30023d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:01.006741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5791eb-ba24-4717-8649-519b2e30023d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:01.006979 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5791eb-ba24-4717-8649-519b2e30023d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:01.010345 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 23/49] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:07:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:01.021323 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f3ff3-2427-4520-bf5d-871b13665cec None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:01.030109 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c80d96-0a78-43fa-804d-818fb4985c8a None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:01.030521 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c80d96-0a78-43fa-804d-818fb4985c8a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:01.030870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c80d96-0a78-43fa-804d-818fb4985c8a None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:01.032756 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 24/50] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:07:01.044433 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f38ab2c-e9a9-4ffa-b3d2-b2078f46c549 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:01.044744 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f38ab2c-e9a9-4ffa-b3d2-b2078f46c549 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:01.045177 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f38ab2c-e9a9-4ffa-b3d2-b2078f46c549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:01.071229 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-9f38ab2c-e9a9-4ffa-b3d2-b2078f46c549 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:01.072592 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9f38ab2c-e9a9-4ffa-b3d2-b2078f46c549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:01.190342 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 25/51] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 19:07:01.216271 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f3ff3-2427-4520-bf5d-871b13665cec None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:01.220359 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f38ab2c-e9a9-4ffa-b3d2-b2078f46c549 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:01.256605 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:01.277838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:01.278998 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6072a970-2b80-401a-85d9-877929bfe2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:01.280027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072a970-2b80-401a-85d9-877929bfe2e3 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:01.280495 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072a970-2b80-401a-85d9-877929bfe2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:01.280967 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072a970-2b80-401a-85d9-877929bfe2e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:01.282683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6072a970-2b80-401a-85d9-877929bfe2e3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:01.287189 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6072a970-2b80-401a-85d9-877929bfe2e3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:01.349002 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 27/52] 10.222.0.18 () {60 vars in 1140 bytes} [Mon Jul 25 19:07:01 2022] POST /identity/v3/projects => generated 298 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:07:01.447799 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f38ab2c-e9a9-4ffa-b3d2-b2078f46c549 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:01.522210 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a1c15-fe65-44f0-9ecf-0d881abd1abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:01.523049 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a1c15-fe65-44f0-9ecf-0d881abd1abb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:01.523355 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a1c15-fe65-44f0-9ecf-0d881abd1abb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:01.523679 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a1c15-fe65-44f0-9ecf-0d881abd1abb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:01.542517 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a1c15-fe65-44f0-9ecf-0d881abd1abb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:07:01.552804 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-825a1c15-fe65-44f0-9ecf-0d881abd1abb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:07:01.552804 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:07:01.552804 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:07:01.552804 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:07:01.552804 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:07:01.552804 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:07:01.552804 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:07:01.552804 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:07:01.552804 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:07:01.552804 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:07:01.552804 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:07:01.552804 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:07:01.552804 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:07:01.552804 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:07:01.552804 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:07:01.552804 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:07:01.554676 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:07:01.554676 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:07:01.554676 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:07:01.554676 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:07:01.554676 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:07:01.554676 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:07:01.554676 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:07:01.554676 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:07:01.554676 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:07:01.554676 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:07:01.554676 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:07:01.554676 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:07:01.554676 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:07:01.554676 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:07:01.554676 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:07:01.554676 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:07:01.556471 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:07:01.556471 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:07:01.556471 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:07:01.556471 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:07:01.556471 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:07:01.556471 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:07:01.556471 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:07:01.556471 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:07:01.556471 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:07:01.556471 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:07:01.556471 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:07:01.556471 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:07:01.556471 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 26/53] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Jul 25 19:07:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:07:01.565148 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6072a970-2b80-401a-85d9-877929bfe2e3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:01.729051 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6072a970-2b80-401a-85d9-877929bfe2e3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:01.779199 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78714a-32fd-405c-b7ec-9330f6aeb7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:01.780387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78714a-32fd-405c-b7ec-9330f6aeb7f0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:01.780894 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78714a-32fd-405c-b7ec-9330f6aeb7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:01.782294 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78714a-32fd-405c-b7ec-9330f6aeb7f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:01.784026 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c78714a-32fd-405c-b7ec-9330f6aeb7f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:01.784357 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c78714a-32fd-405c-b7ec-9330f6aeb7f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:01.786060 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c78714a-32fd-405c-b7ec-9330f6aeb7f0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:01.798437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 28/54] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:07:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:01.804810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a1c15-fe65-44f0-9ecf-0d881abd1abb None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:01.940975 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:01.980004 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a1c15-fe65-44f0-9ecf-0d881abd1abb None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:02.053128 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6777f04-a1f0-4c65-a578-852433dd7599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:02.054683 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6777f04-a1f0-4c65-a578-852433dd7599 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:02.055188 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6777f04-a1f0-4c65-a578-852433dd7599 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:02.055748 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6777f04-a1f0-4c65-a578-852433dd7599 None admin] PATH_INFO: `/v3/users/35d700f1abf943789efbf8a08bd9b587` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:02.058274 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6777f04-a1f0-4c65-a578-852433dd7599 None admin] RBAC: Authorizing `identity:get_user(user_id=35d700f1abf943789efbf8a08bd9b587)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:02.061193 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6777f04-a1f0-4c65-a578-852433dd7599 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:02.065304 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 27/55] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:07:01 2022] GET /identity/v3/users/35d700f1abf943789efbf8a08bd9b587 => generated 247 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:02.074702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78714a-32fd-405c-b7ec-9330f6aeb7f0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:02.162226 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:02.290441 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78714a-32fd-405c-b7ec-9330f6aeb7f0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:02.358587 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f996ca5-5d3a-4cab-967a-eaa49f08491c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:02.360422 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f996ca5-5d3a-4cab-967a-eaa49f08491c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:02.360678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f996ca5-5d3a-4cab-967a-eaa49f08491c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:02.361075 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f996ca5-5d3a-4cab-967a-eaa49f08491c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:02.363503 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f996ca5-5d3a-4cab-967a-eaa49f08491c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:02.366352 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f996ca5-5d3a-4cab-967a-eaa49f08491c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:02.369670 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 29/56] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:07:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:02.379578 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6777f04-a1f0-4c65-a578-852433dd7599 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:02.597849 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6777f04-a1f0-4c65-a578-852433dd7599 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:02.682084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f1a6b-a3ce-4ee8-8659-4b80045c0768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:02.683221 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f1a6b-a3ce-4ee8-8659-4b80045c0768 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:02.683770 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f1a6b-a3ce-4ee8-8659-4b80045c0768 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:02.684292 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f1a6b-a3ce-4ee8-8659-4b80045c0768 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:02.686009 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f1a6b-a3ce-4ee8-8659-4b80045c0768 None admin] RBAC: Adding query filter params (role.id=ed7fec18968241efa76a0ee769d96da9, scope.domain.id=default, user.id=35d700f1abf943789efbf8a08bd9b587) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:02.686590 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f1a6b-a3ce-4ee8-8659-4b80045c0768 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:02.690315 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f1a6b-a3ce-4ee8-8659-4b80045c0768 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:02.706144 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 28/57] 10.222.0.18 () {56 vars in 1322 bytes} [Mon Jul 25 19:07:02 2022] GET /identity/v3/role_assignments?user.id=35d700f1abf943789efbf8a08bd9b587&scope.domain.id=default&role.id=ed7fec18968241efa76a0ee769d96da9 => generated 530 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:04.114499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1960887-697d-471e-9aaa-9276cc38e239 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:04.115184 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1960887-697d-471e-9aaa-9276cc38e239 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:04.115576 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1960887-697d-471e-9aaa-9276cc38e239 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:04.118386 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 30/58] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:07:04.126942 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e861a-7b35-4ec0-a48f-9891208f52ef None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:04.127729 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e861a-7b35-4ec0-a48f-9891208f52ef None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:04.127729 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e861a-7b35-4ec0-a48f-9891208f52ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:04.161262 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-916e861a-7b35-4ec0-a48f-9891208f52ef None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:04.163225 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-916e861a-7b35-4ec0-a48f-9891208f52ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:04.184590 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:04.292162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 29/59] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:07:04.339939 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1960887-697d-471e-9aaa-9276cc38e239 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:04.522239 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1960887-697d-471e-9aaa-9276cc38e239 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:04.533652 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:04.579916 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b4cd88-cd9c-4c86-9a4a-59a1082eea04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:04.580900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4cd88-cd9c-4c86-9a4a-59a1082eea04 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:04.581278 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4cd88-cd9c-4c86-9a4a-59a1082eea04 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:04.581646 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4cd88-cd9c-4c86-9a4a-59a1082eea04 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:04.583163 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b4cd88-cd9c-4c86-9a4a-59a1082eea04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:04.584746 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b4cd88-cd9c-4c86-9a4a-59a1082eea04 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:04.586562 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 31/60] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:07:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:04.593793 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e861a-7b35-4ec0-a48f-9891208f52ef None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:04.828384 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e861a-7b35-4ec0-a48f-9891208f52ef None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:04.903001 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e74f80-05fb-4494-8ce2-be512b5c85be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:04.903905 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e74f80-05fb-4494-8ce2-be512b5c85be None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:04.904392 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e74f80-05fb-4494-8ce2-be512b5c85be None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:04.904827 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e74f80-05fb-4494-8ce2-be512b5c85be None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:04.905873 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e74f80-05fb-4494-8ce2-be512b5c85be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:04.908302 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e74f80-05fb-4494-8ce2-be512b5c85be None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:04.950077 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 30/61] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Jul 25 19:07:04 2022] POST /identity/v3/users => generated 275 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:07:09.837324 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d5882-0307-4858-8940-428f5a25271e None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:09.838151 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d5882-0307-4858-8940-428f5a25271e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:09.838279 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d5882-0307-4858-8940-428f5a25271e None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:09.841369 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 32/62] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:07:09.850853 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4e96b-554a-4e84-b371-cd97768bdbbe None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:09.852227 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4e96b-554a-4e84-b371-cd97768bdbbe None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:09.853138 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4e96b-554a-4e84-b371-cd97768bdbbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:09.870597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:09.891963 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5ce4e96b-554a-4e84-b371-cd97768bdbbe None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:09.893915 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5ce4e96b-554a-4e84-b371-cd97768bdbbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:10.071879 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 31/63] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:07:10.110496 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e6341-28a3-4571-b8e2-4b7816837c96 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:10.111489 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e6341-28a3-4571-b8e2-4b7816837c96 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:10.112331 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e6341-28a3-4571-b8e2-4b7816837c96 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:10.115352 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 33/64] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:07:10.125804 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959553ba-6ae2-4daa-b380-4a9e4b9f2784 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:10.126773 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959553ba-6ae2-4daa-b380-4a9e4b9f2784 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:10.127759 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959553ba-6ae2-4daa-b380-4a9e4b9f2784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:10.132935 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e6341-28a3-4571-b8e2-4b7816837c96 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:10.178608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-959553ba-6ae2-4daa-b380-4a9e4b9f2784 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:10.180844 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-959553ba-6ae2-4daa-b380-4a9e4b9f2784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:10.335256 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 32/65] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:07:10.338246 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5e656-7c5f-4777-bf4d-fb03beedcf8b None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:10.339070 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5e656-7c5f-4777-bf4d-fb03beedcf8b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:10.339807 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5e656-7c5f-4777-bf4d-fb03beedcf8b None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:10.344086 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 33/66] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:07:10.352909 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e0b128-870b-4b7d-a8be-f052eab8af2a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:10.353758 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e0b128-870b-4b7d-a8be-f052eab8af2a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:10.354492 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e0b128-870b-4b7d-a8be-f052eab8af2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:10.371694 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e6341-28a3-4571-b8e2-4b7816837c96 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:10.405139 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-35e0b128-870b-4b7d-a8be-f052eab8af2a None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:10.408439 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-35e0b128-870b-4b7d-a8be-f052eab8af2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:10.462369 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ba7ea-273d-4c16-b631-17406c45745c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:10.464399 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ba7ea-273d-4c16-b631-17406c45745c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:10.465267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ba7ea-273d-4c16-b631-17406c45745c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:10.466007 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ba7ea-273d-4c16-b631-17406c45745c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:10.500382 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ba7ea-273d-4c16-b631-17406c45745c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:07:10.517121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-691ba7ea-273d-4c16-b631-17406c45745c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 19:07:10.517121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:07:10.517121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:07:10.517121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:07:10.517121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:07:10.517121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:07:10.517121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:07:10.517121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:07:10.517121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:07:10.517121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:07:10.517121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:07:10.517121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:07:10.517121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:07:10.517121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:07:10.517121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:07:10.517121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:07:10.518884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:07:10.518884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:07:10.518884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:07:10.518884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:07:10.518884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:07:10.518884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:07:10.518884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:07:10.518884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:07:10.518884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:07:10.518884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:07:10.518884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:07:10.518884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:07:10.518884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:07:10.518884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:07:10.518884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:07:10.518884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:07:10.520480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:07:10.520480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:07:10.520480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:07:10.520480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:07:10.520480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:07:10.520480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:07:10.520480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:07:10.520480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:07:10.520480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:07:10.520480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:07:10.520480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 19:07:10.520480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:07:10.525214 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 34/67] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:07:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:07:10.527238 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ba7ea-273d-4c16-b631-17406c45745c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:10.552218 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:10.575323 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 34/68] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:07:10.575323 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e0b128-870b-4b7d-a8be-f052eab8af2a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:10.620343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:10.785624 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ba7ea-273d-4c16-b631-17406c45745c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:10.796889 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e0b128-870b-4b7d-a8be-f052eab8af2a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:10.862704 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded57fe-cc42-4f81-b024-2ad21c9398fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:10.864273 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded57fe-cc42-4f81-b024-2ad21c9398fd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:10.865083 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded57fe-cc42-4f81-b024-2ad21c9398fd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:10.865591 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded57fe-cc42-4f81-b024-2ad21c9398fd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:10.877146 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2c4de1-d1d1-4854-8f67-547b69ce7457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:10.880500 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c4de1-d1d1-4854-8f67-547b69ce7457 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:10.880500 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c4de1-d1d1-4854-8f67-547b69ce7457 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:10.880500 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c4de1-d1d1-4854-8f67-547b69ce7457 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:10.881235 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2c4de1-d1d1-4854-8f67-547b69ce7457 None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:10.881486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2c4de1-d1d1-4854-8f67-547b69ce7457 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:10.883963 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2c4de1-d1d1-4854-8f67-547b69ce7457 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:10.893021 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eded57fe-cc42-4f81-b024-2ad21c9398fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:07:10.909311 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-eded57fe-cc42-4f81-b024-2ad21c9398fd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 19:07:10.909311 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:07:10.909311 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:07:10.909311 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:07:10.909311 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:07:10.909311 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:07:10.909311 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:07:10.909311 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:07:10.909311 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:07:10.909311 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:07:10.909311 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:07:10.909311 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:07:10.909311 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:07:10.909311 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:07:10.909311 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:07:10.909311 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:07:10.911009 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:07:10.911009 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:07:10.911009 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:07:10.911009 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:07:10.911009 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:07:10.911009 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:07:10.911009 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:07:10.911009 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:07:10.911009 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:07:10.911009 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:07:10.911009 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:07:10.911009 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:07:10.911009 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:07:10.911009 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:07:10.911009 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:07:10.911009 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:07:10.913601 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:07:10.913601 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:07:10.913601 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:07:10.913601 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:07:10.913601 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:07:10.913601 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:07:10.913601 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:07:10.913601 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:07:10.913601 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:07:10.913601 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:07:10.913601 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 19:07:10.913601 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:07:10.919654 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 35/69] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Jul 25 19:07:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:07:10.919654 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3126408-cd8a-4e60-bbec-06046d72bd86 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:10.919654 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3126408-cd8a-4e60-bbec-06046d72bd86 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:10.920901 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3126408-cd8a-4e60-bbec-06046d72bd86 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:10.924153 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 35/70] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Jul 25 19:07:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:10.926483 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 36/71] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:07:10.926483 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3126408-cd8a-4e60-bbec-06046d72bd86 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:10.926832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2c4de1-d1d1-4854-8f67-547b69ce7457 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:11.063621 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:11.122698 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:11.139258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3126408-cd8a-4e60-bbec-06046d72bd86 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:11.157638 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2c4de1-d1d1-4854-8f67-547b69ce7457 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:11.219975 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a06624-4437-42ae-b21f-9b04f2778987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:11.221246 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a06624-4437-42ae-b21f-9b04f2778987 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:11.221753 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a06624-4437-42ae-b21f-9b04f2778987 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:11.222277 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a06624-4437-42ae-b21f-9b04f2778987 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:11.223694 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a06624-4437-42ae-b21f-9b04f2778987 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:11.224299 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a06624-4437-42ae-b21f-9b04f2778987 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:11.225997 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0673a677-10fc-474d-b1dc-92e5331adb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:11.226932 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a06624-4437-42ae-b21f-9b04f2778987 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:11.227267 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0673a677-10fc-474d-b1dc-92e5331adb83 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:11.227817 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0673a677-10fc-474d-b1dc-92e5331adb83 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:11.228329 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0673a677-10fc-474d-b1dc-92e5331adb83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:11.244678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 37/72] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:07:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:11.246931 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedba631-d15d-47b4-93d7-664210c0d3f2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:11.247425 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedba631-d15d-47b4-93d7-664210c0d3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:11.247903 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedba631-d15d-47b4-93d7-664210c0d3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:11.259260 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0673a677-10fc-474d-b1dc-92e5331adb83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:07:11.270982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-0673a677-10fc-474d-b1dc-92e5331adb83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:07:11.270982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:07:11.270982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:07:11.270982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:07:11.270982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:07:11.270982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:07:11.270982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:07:11.270982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:07:11.270982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:07:11.270982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:07:11.270982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:07:11.270982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:07:11.270982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:07:11.270982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:07:11.270982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:07:11.270982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:07:11.272850 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:07:11.272850 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:07:11.272850 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:07:11.272850 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:07:11.272850 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:07:11.272850 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:07:11.272850 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:07:11.272850 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:07:11.272850 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:07:11.272850 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:07:11.272850 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:07:11.272850 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:07:11.272850 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:07:11.272850 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:07:11.272850 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:07:11.272850 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:07:11.274301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:07:11.274301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:07:11.274301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:07:11.274301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:07:11.274301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:07:11.274301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:07:11.274301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:07:11.274301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:07:11.274301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:07:11.274301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:07:11.274301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:07:11.274301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:07:11.274301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 36/73] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Jul 25 19:07:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:07:11.275466 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0673a677-10fc-474d-b1dc-92e5331adb83 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:11.286782 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-aedba631-d15d-47b4-93d7-664210c0d3f2 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:11.288850 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-aedba631-d15d-47b4-93d7-664210c0d3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:11.429633 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 38/74] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:07:11.431102 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362ff64-0590-42f6-a1e4-d3a7fd43c3c0 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:11.431418 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362ff64-0590-42f6-a1e4-d3a7fd43c3c0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:11.431719 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362ff64-0590-42f6-a1e4-d3a7fd43c3c0 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:11.435583 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 39/75] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:07:11.435847 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c362ff64-0590-42f6-a1e4-d3a7fd43c3c0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:11.463834 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0673a677-10fc-474d-b1dc-92e5331adb83 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:11.498352 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:11.527358 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb80b6-7c4f-44cb-996c-9bffe513b8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:11.528539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb80b6-7c4f-44cb-996c-9bffe513b8f4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:11.529010 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb80b6-7c4f-44cb-996c-9bffe513b8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:11.529386 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb80b6-7c4f-44cb-996c-9bffe513b8f4 None admin] PATH_INFO: `/v3/users/1ee13b9555134b9cafac7f5f0a47204b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:11.553820 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eb80b6-7c4f-44cb-996c-9bffe513b8f4 None admin] RBAC: Authorizing `identity:get_user(user_id=1ee13b9555134b9cafac7f5f0a47204b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:11.556491 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eb80b6-7c4f-44cb-996c-9bffe513b8f4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:11.559853 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 37/76] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:07:11 2022] GET /identity/v3/users/1ee13b9555134b9cafac7f5f0a47204b => generated 275 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:11.561516 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb80b6-7c4f-44cb-996c-9bffe513b8f4 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:11.630765 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:11.654394 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c362ff64-0590-42f6-a1e4-d3a7fd43c3c0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:11.705127 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc4f3b-7e2c-4f6b-85c3-31cecac9f2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:11.706037 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc4f3b-7e2c-4f6b-85c3-31cecac9f2ca None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:11.706393 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc4f3b-7e2c-4f6b-85c3-31cecac9f2ca None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:11.706847 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc4f3b-7e2c-4f6b-85c3-31cecac9f2ca None admin] PATH_INFO: `/v3/users/1ee13b9555134b9cafac7f5f0a47204b` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:11.708759 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bc4f3b-7e2c-4f6b-85c3-31cecac9f2ca None admin] RBAC: Authorizing `identity:get_user(user_id=1ee13b9555134b9cafac7f5f0a47204b)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:11.710686 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bc4f3b-7e2c-4f6b-85c3-31cecac9f2ca None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:11.712835 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 40/77] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:07:11 2022] GET /identity/v3/users/1ee13b9555134b9cafac7f5f0a47204b => generated 275 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:11.714653 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d38fd-453d-4edb-8d86-a4a4ac621b06 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:11.714653 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d38fd-453d-4edb-8d86-a4a4ac621b06 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:11.714920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d38fd-453d-4edb-8d86-a4a4ac621b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:11.736837 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0b5d38fd-453d-4edb-8d86-a4a4ac621b06 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:11.738496 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0b5d38fd-453d-4edb-8d86-a4a4ac621b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:11.778330 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb80b6-7c4f-44cb-996c-9bffe513b8f4 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:11.842270 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e81e9-bdc7-42e3-997d-0942508b108c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:11.844120 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e81e9-bdc7-42e3-997d-0942508b108c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:11.844665 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e81e9-bdc7-42e3-997d-0942508b108c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:11.845458 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e81e9-bdc7-42e3-997d-0942508b108c None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:11.846938 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e81e9-bdc7-42e3-997d-0942508b108c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:11.847487 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e81e9-bdc7-42e3-997d-0942508b108c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:11.850212 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e81e9-bdc7-42e3-997d-0942508b108c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:11.851916 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 41/78] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:07:11.853044 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d38fd-453d-4edb-8d86-a4a4ac621b06 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:11.876596 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 38/79] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:07:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:11.877748 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e81e9-bdc7-42e3-997d-0942508b108c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:11.953560 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:12.017989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:12.060663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e81e9-bdc7-42e3-997d-0942508b108c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:12.066574 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d38fd-453d-4edb-8d86-a4a4ac621b06 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:12.138405 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e046cd18-4fd5-49c4-ad5c-d6cf1fab8264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:12.139630 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046cd18-4fd5-49c4-ad5c-d6cf1fab8264 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:12.140697 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046cd18-4fd5-49c4-ad5c-d6cf1fab8264 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:12.141204 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046cd18-4fd5-49c4-ad5c-d6cf1fab8264 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:12.169242 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fb0963-9cc1-4b30-9d4e-b5a0e1af35cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:12.170271 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb0963-9cc1-4b30-9d4e-b5a0e1af35cc None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:12.170580 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb0963-9cc1-4b30-9d4e-b5a0e1af35cc None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:12.170859 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb0963-9cc1-4b30-9d4e-b5a0e1af35cc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:12.173265 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fb0963-9cc1-4b30-9d4e-b5a0e1af35cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:12.174897 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fb0963-9cc1-4b30-9d4e-b5a0e1af35cc None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:12.176803 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 42/80] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:07:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:12.177647 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fb0963-9cc1-4b30-9d4e-b5a0e1af35cc None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:12.282819 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e046cd18-4fd5-49c4-ad5c-d6cf1fab8264 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:12.285662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e046cd18-4fd5-49c4-ad5c-d6cf1fab8264 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:12.358513 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fb0963-9cc1-4b30-9d4e-b5a0e1af35cc None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:12.359659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:12.417646 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87492a7-900b-447c-89e5-a63a8d0c81d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:12.418685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87492a7-900b-447c-89e5-a63a8d0c81d1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:12.419078 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87492a7-900b-447c-89e5-a63a8d0c81d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:12.419415 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87492a7-900b-447c-89e5-a63a8d0c81d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:12.463236 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:12.501664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 39/81] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:07:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 19:07:12.503566 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e046cd18-4fd5-49c4-ad5c-d6cf1fab8264 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:12.548305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87492a7-900b-447c-89e5-a63a8d0c81d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:12.551030 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87492a7-900b-447c-89e5-a63a8d0c81d1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:12.711988 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e046cd18-4fd5-49c4-ad5c-d6cf1fab8264 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:12.728686 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 43/82] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:07:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 19:07:12.729828 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87492a7-900b-447c-89e5-a63a8d0c81d1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:12.773613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c52d91e-d7c6-408d-99d9-011dd2170184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:12.774554 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52d91e-d7c6-408d-99d9-011dd2170184 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:12.774842 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52d91e-d7c6-408d-99d9-011dd2170184 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:12.775129 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52d91e-d7c6-408d-99d9-011dd2170184 None admin] PATH_INFO: `/v3/users/35d700f1abf943789efbf8a08bd9b587` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:12.776898 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c52d91e-d7c6-408d-99d9-011dd2170184 None admin] RBAC: Authorizing `identity:get_user(user_id=35d700f1abf943789efbf8a08bd9b587)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:12.778838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c52d91e-d7c6-408d-99d9-011dd2170184 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:12.780830 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 40/83] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:07:12 2022] GET /identity/v3/users/35d700f1abf943789efbf8a08bd9b587 => generated 247 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:12.781716 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c52d91e-d7c6-408d-99d9-011dd2170184 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:12.788697 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:12.813775 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:12.894941 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87492a7-900b-447c-89e5-a63a8d0c81d1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:12.958960 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c52d91e-d7c6-408d-99d9-011dd2170184 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:12.961649 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de07217-cd6b-4fc3-a117-50557792881e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:12.962748 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de07217-cd6b-4fc3-a117-50557792881e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:12.963225 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de07217-cd6b-4fc3-a117-50557792881e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:12.963731 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de07217-cd6b-4fc3-a117-50557792881e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:12.979114 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de07217-cd6b-4fc3-a117-50557792881e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:07:12.988944 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-2de07217-cd6b-4fc3-a117-50557792881e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 19:07:12.988944 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:07:12.988944 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:07:12.988944 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:07:12.988944 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:07:12.988944 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:07:12.988944 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:07:12.988944 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:07:12.988944 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:07:12.988944 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:07:12.988944 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:07:12.988944 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:07:12.988944 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:07:12.988944 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:07:12.988944 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:07:12.988944 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:07:12.990918 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:07:12.990918 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:07:12.990918 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:07:12.990918 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:07:12.990918 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:07:12.990918 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:07:12.990918 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:07:12.990918 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:07:12.990918 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:07:12.990918 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:07:12.990918 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:07:12.990918 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:07:12.990918 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:07:12.990918 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:07:12.990918 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:07:12.990918 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:07:12.992680 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:07:12.992680 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:07:12.992680 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:07:12.992680 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:07:12.992680 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:07:12.992680 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:07:12.992680 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:07:12.992680 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:07:12.992680 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:07:12.992680 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:07:12.992680 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 19:07:12.992680 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:07:12.992680 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 44/84] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:07:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:07:12.992680 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de07217-cd6b-4fc3-a117-50557792881e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:13.017459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09494a88-84e4-4623-bb1f-9101620f726c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:13.018295 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09494a88-84e4-4623-bb1f-9101620f726c None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:13.018708 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09494a88-84e4-4623-bb1f-9101620f726c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:13.019088 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09494a88-84e4-4623-bb1f-9101620f726c None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:13.019991 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09494a88-84e4-4623-bb1f-9101620f726c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:13.021825 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09494a88-84e4-4623-bb1f-9101620f726c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:13.046428 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 41/85] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Jul 25 19:07:12 2022] POST /identity/v3/users => generated 289 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:07:13.047794 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09494a88-84e4-4623-bb1f-9101620f726c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:13.184670 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:13.233635 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de07217-cd6b-4fc3-a117-50557792881e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:13.270296 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:13.286006 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09494a88-84e4-4623-bb1f-9101620f726c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:13.335038 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6954f9-1571-47ef-9ae6-2a832b12bcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:13.336689 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6954f9-1571-47ef-9ae6-2a832b12bcf0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:13.337187 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6954f9-1571-47ef-9ae6-2a832b12bcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:13.337682 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6954f9-1571-47ef-9ae6-2a832b12bcf0 None admin] PATH_INFO: `/v3/projects/09f6174e875f4cf58a2eb5019ef11189` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:13.340142 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6954f9-1571-47ef-9ae6-2a832b12bcf0 None admin] RBAC: Authorizing `identity:get_project(project_id=09f6174e875f4cf58a2eb5019ef11189)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:13.356423 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6954f9-1571-47ef-9ae6-2a832b12bcf0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:13.359046 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 45/86] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:07:12 2022] GET /identity/v3/projects/09f6174e875f4cf58a2eb5019ef11189 => generated 298 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:13.360070 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6954f9-1571-47ef-9ae6-2a832b12bcf0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:13.363107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac710433-f39a-4684-b0d1-0bb9d15ccb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:13.364439 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac710433-f39a-4684-b0d1-0bb9d15ccb55 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:13.364876 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac710433-f39a-4684-b0d1-0bb9d15ccb55 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:13.365348 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac710433-f39a-4684-b0d1-0bb9d15ccb55 None admin] PATH_INFO: `/v3/projects/09f6174e875f4cf58a2eb5019ef11189` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:13.367063 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac710433-f39a-4684-b0d1-0bb9d15ccb55 None admin] RBAC: Authorizing `identity:get_project(project_id=09f6174e875f4cf58a2eb5019ef11189)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:13.369758 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac710433-f39a-4684-b0d1-0bb9d15ccb55 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:13.373065 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 42/87] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:07:13 2022] GET /identity/v3/projects/09f6174e875f4cf58a2eb5019ef11189 => generated 298 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:13.374592 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac710433-f39a-4684-b0d1-0bb9d15ccb55 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:13.608076 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac710433-f39a-4684-b0d1-0bb9d15ccb55 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:13.614262 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6954f9-1571-47ef-9ae6-2a832b12bcf0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:13.662794 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:13.680580 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa719bab-ad39-448c-a28e-2054c7b19900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:13.680580 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa719bab-ad39-448c-a28e-2054c7b19900 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:13.681008 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa719bab-ad39-448c-a28e-2054c7b19900 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:13.681708 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa719bab-ad39-448c-a28e-2054c7b19900 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:13.683242 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa719bab-ad39-448c-a28e-2054c7b19900 None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:13.683936 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa719bab-ad39-448c-a28e-2054c7b19900 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:13.687212 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa719bab-ad39-448c-a28e-2054c7b19900 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:13.700048 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 43/88] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Jul 25 19:07:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:13.708140 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa719bab-ad39-448c-a28e-2054c7b19900 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:13.724573 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889caa1d-ba51-4581-9afc-a8bf11d38ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:13.727483 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889caa1d-ba51-4581-9afc-a8bf11d38ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:13.728289 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889caa1d-ba51-4581-9afc-a8bf11d38ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:13.729923 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889caa1d-ba51-4581-9afc-a8bf11d38ec5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:13.854949 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:13.896575 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889caa1d-ba51-4581-9afc-a8bf11d38ec5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:13.899573 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889caa1d-ba51-4581-9afc-a8bf11d38ec5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:13.917694 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa719bab-ad39-448c-a28e-2054c7b19900 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:13.996229 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a131ef-93b2-43e6-bd5f-773a1e3b9b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:14.000139 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a131ef-93b2-43e6-bd5f-773a1e3b9b3f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:14.000679 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a131ef-93b2-43e6-bd5f-773a1e3b9b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:14.001157 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a131ef-93b2-43e6-bd5f-773a1e3b9b3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:14.002722 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a131ef-93b2-43e6-bd5f-773a1e3b9b3f None admin] RBAC: Adding query filter params (role.id=c241bd9634e24bcbb1854c7621d19f81, scope.project.id=09f6174e875f4cf58a2eb5019ef11189, user.id=1ee13b9555134b9cafac7f5f0a47204b) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:14.003343 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a131ef-93b2-43e6-bd5f-773a1e3b9b3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:14.006180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a131ef-93b2-43e6-bd5f-773a1e3b9b3f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:14.023375 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 44/89] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:07:13 2022] GET /identity/v3/role_assignments?user.id=1ee13b9555134b9cafac7f5f0a47204b&scope.project.id=09f6174e875f4cf58a2eb5019ef11189&role.id=c241bd9634e24bcbb1854c7621d19f81 => generated 259 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:14.023375 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a131ef-93b2-43e6-bd5f-773a1e3b9b3f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:14.166558 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:14.172081 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 46/90] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:07:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 812 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 19:07:14.173678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889caa1d-ba51-4581-9afc-a8bf11d38ec5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:14.265090 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a131ef-93b2-43e6-bd5f-773a1e3b9b3f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:14.331169 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:14.342732 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b7d0c2-c6d2-461e-af3d-ced6a62bbcd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:14.343816 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b7d0c2-c6d2-461e-af3d-ced6a62bbcd1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:14.344291 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b7d0c2-c6d2-461e-af3d-ced6a62bbcd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:14.344741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b7d0c2-c6d2-461e-af3d-ced6a62bbcd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:14.346173 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b7d0c2-c6d2-461e-af3d-ced6a62bbcd1 None admin] RBAC: Adding query filter params (role.id=191e7e31dad148b7a0eb52d3afb5e50e, scope.project.id=09f6174e875f4cf58a2eb5019ef11189, user.id=1ee13b9555134b9cafac7f5f0a47204b) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:14.346737 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b7d0c2-c6d2-461e-af3d-ced6a62bbcd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:14.349015 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b7d0c2-c6d2-461e-af3d-ced6a62bbcd1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:14.364032 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 45/91] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:07:14 2022] GET /identity/v3/role_assignments?user.id=1ee13b9555134b9cafac7f5f0a47204b&scope.project.id=09f6174e875f4cf58a2eb5019ef11189&role.id=191e7e31dad148b7a0eb52d3afb5e50e => generated 259 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:14.369662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b7d0c2-c6d2-461e-af3d-ced6a62bbcd1 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:14.401040 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889caa1d-ba51-4581-9afc-a8bf11d38ec5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:14.492994 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca08a9-83f8-470c-b260-d30786afd5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:14.494551 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca08a9-83f8-470c-b260-d30786afd5e6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:14.495038 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca08a9-83f8-470c-b260-d30786afd5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:14.495551 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca08a9-83f8-470c-b260-d30786afd5e6 None admin] PATH_INFO: `/v3/users/1ee13b9555134b9cafac7f5f0a47204b` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:14.503211 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cca08a9-83f8-470c-b260-d30786afd5e6 None admin] RBAC: Authorizing `identity:get_user(user_id=1ee13b9555134b9cafac7f5f0a47204b)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:14.506270 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cca08a9-83f8-470c-b260-d30786afd5e6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:14.509725 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 47/92] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:07:14 2022] GET /identity/v3/users/1ee13b9555134b9cafac7f5f0a47204b => generated 275 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:14.530306 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca08a9-83f8-470c-b260-d30786afd5e6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:14.815900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b7d0c2-c6d2-461e-af3d-ced6a62bbcd1 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:14.882943 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:14.948694 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca08a9-83f8-470c-b260-d30786afd5e6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:14.973934 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a870f3-0c96-4295-b20b-d70069ace896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:14.975584 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a870f3-0c96-4295-b20b-d70069ace896 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:14.976105 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a870f3-0c96-4295-b20b-d70069ace896 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:14.976660 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a870f3-0c96-4295-b20b-d70069ace896 None admin] PATH_INFO: `/v3/projects/09f6174e875f4cf58a2eb5019ef11189` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:14.985166 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a870f3-0c96-4295-b20b-d70069ace896 None admin] RBAC: Authorizing `identity:get_project(project_id=09f6174e875f4cf58a2eb5019ef11189)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:14.988144 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a870f3-0c96-4295-b20b-d70069ace896 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:14.991729 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 46/93] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:07:14 2022] GET /identity/v3/projects/09f6174e875f4cf58a2eb5019ef11189 => generated 298 bytes in 623 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:15.017473 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a870f3-0c96-4295-b20b-d70069ace896 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:15.093242 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ed171-34cc-4a1e-9cf2-2085f48b7453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:15.094331 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ed171-34cc-4a1e-9cf2-2085f48b7453 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:15.094796 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ed171-34cc-4a1e-9cf2-2085f48b7453 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:15.095302 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ed171-34cc-4a1e-9cf2-2085f48b7453 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:15.160157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:15.346831 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19ed171-34cc-4a1e-9cf2-2085f48b7453 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:15.349772 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19ed171-34cc-4a1e-9cf2-2085f48b7453 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:15.479031 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a870f3-0c96-4295-b20b-d70069ace896 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:15.575708 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20640bdf-0193-46b4-ba57-4e557ccd2333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:15.576873 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20640bdf-0193-46b4-ba57-4e557ccd2333 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:15.581324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20640bdf-0193-46b4-ba57-4e557ccd2333 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:15.582124 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20640bdf-0193-46b4-ba57-4e557ccd2333 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:15.583714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20640bdf-0193-46b4-ba57-4e557ccd2333 None admin] RBAC: Adding query filter params (role.id=ed7fec18968241efa76a0ee769d96da9, scope.project.id=09f6174e875f4cf58a2eb5019ef11189, user.id=35d700f1abf943789efbf8a08bd9b587) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:15.584640 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20640bdf-0193-46b4-ba57-4e557ccd2333 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:15.587741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20640bdf-0193-46b4-ba57-4e557ccd2333 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:15.600810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 47/94] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:07:15 2022] GET /identity/v3/role_assignments?user.id=35d700f1abf943789efbf8a08bd9b587&scope.project.id=09f6174e875f4cf58a2eb5019ef11189&role.id=ed7fec18968241efa76a0ee769d96da9 => generated 259 bytes in 585 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:15.635300 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 48/95] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:07:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 1109 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 19:07:15.645356 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20640bdf-0193-46b4-ba57-4e557ccd2333 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:15.729903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:15.836603 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20640bdf-0193-46b4-ba57-4e557ccd2333 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:15.918969 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380ce12b-bcf0-428b-b631-d050e97ce293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:15.925339 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ce12b-bcf0-428b-b631-d050e97ce293 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:15.926264 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ce12b-bcf0-428b-b631-d050e97ce293 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:15.927025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ce12b-bcf0-428b-b631-d050e97ce293 None admin] PATH_INFO: `/v3/projects/c1814fe30f064bc1875e3a1eac8ed490` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:15.929995 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380ce12b-bcf0-428b-b631-d050e97ce293 None admin] RBAC: Authorizing `identity:get_project(project_id=c1814fe30f064bc1875e3a1eac8ed490)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:15.933567 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380ce12b-bcf0-428b-b631-d050e97ce293 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:15.938827 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 48/96] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:07:15 2022] GET /identity/v3/projects/c1814fe30f064bc1875e3a1eac8ed490 => generated 312 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:07:15.949370 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ed171-34cc-4a1e-9cf2-2085f48b7453 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:16.060155 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:16.180819 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ed171-34cc-4a1e-9cf2-2085f48b7453 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:16.249302 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc3e4f-394a-4e5a-a535-51980a808280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:16.251659 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc3e4f-394a-4e5a-a535-51980a808280 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:16.252108 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc3e4f-394a-4e5a-a535-51980a808280 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:16.252542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc3e4f-394a-4e5a-a535-51980a808280 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:16.253782 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc3e4f-394a-4e5a-a535-51980a808280 None admin] RBAC: Adding query filter params (role.id=c241bd9634e24bcbb1854c7621d19f81, scope.project.id=c1814fe30f064bc1875e3a1eac8ed490, user.id=1ee13b9555134b9cafac7f5f0a47204b) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:16.254361 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc3e4f-394a-4e5a-a535-51980a808280 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:16.256529 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc3e4f-394a-4e5a-a535-51980a808280 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:16.264749 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 49/97] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:07:15 2022] GET /identity/v3/role_assignments?user.id=1ee13b9555134b9cafac7f5f0a47204b&scope.project.id=c1814fe30f064bc1875e3a1eac8ed490&role.id=c241bd9634e24bcbb1854c7621d19f81 => generated 259 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:17.398599 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38f630-0907-41e3-a862-7bb821643bfe None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:17.400429 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38f630-0907-41e3-a862-7bb821643bfe None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:17.401325 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d38f630-0907-41e3-a862-7bb821643bfe None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:17.404775 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 49/98] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:07:17.414160 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e11e29-85c9-4200-88f1-788782dc02ac None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:17.415803 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e11e29-85c9-4200-88f1-788782dc02ac None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:17.416627 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e11e29-85c9-4200-88f1-788782dc02ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:17.460343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b1e11e29-85c9-4200-88f1-788782dc02ac None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:17.462586 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b1e11e29-85c9-4200-88f1-788782dc02ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:17.649651 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 50/99] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:07:17.721787 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d38f630-0907-41e3-a862-7bb821643bfe None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:17.957052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d38f630-0907-41e3-a862-7bb821643bfe None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:17.973855 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620bba8-84a6-48b1-8346-6aae71b32e02 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:17.974281 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620bba8-84a6-48b1-8346-6aae71b32e02 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:17.974759 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620bba8-84a6-48b1-8346-6aae71b32e02 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:17.983060 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 51/100] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:17 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:07:17.998385 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a8f0b-a04c-4b6b-9436-79e6c82454d1 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:17.998902 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a8f0b-a04c-4b6b-9436-79e6c82454d1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:18.000247 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a8f0b-a04c-4b6b-9436-79e6c82454d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:18.020319 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:18.081807 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e82a8f0b-a04c-4b6b-9436-79e6c82454d1 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:18.084164 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e82a8f0b-a04c-4b6b-9436-79e6c82454d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:18.109075 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcdc468-e426-4b0c-b509-9bed45abb9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:18.110786 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdc468-e426-4b0c-b509-9bed45abb9e4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:18.111088 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdc468-e426-4b0c-b509-9bed45abb9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:18.111463 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdc468-e426-4b0c-b509-9bed45abb9e4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:18.129900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcdc468-e426-4b0c-b509-9bed45abb9e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:07:18.147087 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-efcdc468-e426-4b0c-b509-9bed45abb9e4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 19:07:18.147087 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:07:18.147087 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:07:18.147087 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:07:18.147087 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:07:18.147087 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:07:18.147087 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:07:18.147087 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:07:18.147087 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:07:18.147087 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:07:18.147087 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:07:18.147087 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:07:18.147087 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:07:18.147087 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:07:18.147087 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:07:18.147087 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:07:18.148825 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:07:18.148825 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:07:18.148825 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:07:18.148825 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:07:18.148825 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:07:18.148825 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:07:18.148825 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:07:18.148825 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:07:18.148825 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:07:18.148825 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:07:18.148825 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:07:18.148825 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:07:18.148825 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:07:18.148825 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:07:18.148825 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:07:18.148825 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:07:18.151101 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:07:18.151101 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:07:18.151101 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:07:18.151101 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:07:18.151101 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:07:18.151101 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:07:18.151101 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:07:18.151101 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:07:18.151101 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:07:18.151101 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:07:18.151101 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 19:07:18.151101 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:07:18.151101 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 50/101] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:07:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 428 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:07:18.159403 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcdc468-e426-4b0c-b509-9bed45abb9e4 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:18.274999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:18.401374 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 52/102] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 19:07:18.487187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcdc468-e426-4b0c-b509-9bed45abb9e4 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:18.489477 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a8f0b-a04c-4b6b-9436-79e6c82454d1 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:18.558584 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3965a75-b01e-4145-9ee4-b4fcdeccb015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:18.559846 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3965a75-b01e-4145-9ee4-b4fcdeccb015 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:18.560366 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3965a75-b01e-4145-9ee4-b4fcdeccb015 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:18.560938 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3965a75-b01e-4145-9ee4-b4fcdeccb015 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:18.563002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3965a75-b01e-4145-9ee4-b4fcdeccb015 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:18.563577 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3965a75-b01e-4145-9ee4-b4fcdeccb015 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:18.565640 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3965a75-b01e-4145-9ee4-b4fcdeccb015 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:18.584297 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 51/103] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Jul 25 19:07:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 426 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:18.606870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3965a75-b01e-4145-9ee4-b4fcdeccb015 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:18.756133 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:18.932367 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3965a75-b01e-4145-9ee4-b4fcdeccb015 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:19.058566 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2825728-5fec-49d4-9b41-d1bd1dd104a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:19.060352 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2825728-5fec-49d4-9b41-d1bd1dd104a0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:19.060901 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2825728-5fec-49d4-9b41-d1bd1dd104a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:19.061792 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2825728-5fec-49d4-9b41-d1bd1dd104a0 None admin] PATH_INFO: `/v3/users/7fc422bf660c4fd1b37fbcc1488b9173` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:19.078239 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a8f0b-a04c-4b6b-9436-79e6c82454d1 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:19.107450 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2825728-5fec-49d4-9b41-d1bd1dd104a0 None admin] RBAC: Authorizing `identity:get_user(user_id=7fc422bf660c4fd1b37fbcc1488b9173)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:19.111131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2825728-5fec-49d4-9b41-d1bd1dd104a0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:19.113312 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 52/104] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:07:18 2022] GET /identity/v3/users/7fc422bf660c4fd1b37fbcc1488b9173 => generated 289 bytes in 507 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:07:19.128380 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2825728-5fec-49d4-9b41-d1bd1dd104a0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:19.292530 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7da425-0031-4abc-baf4-1698be96568e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:19.293591 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7da425-0031-4abc-baf4-1698be96568e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:19.293967 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7da425-0031-4abc-baf4-1698be96568e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:19.294379 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7da425-0031-4abc-baf4-1698be96568e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:19.301100 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7da425-0031-4abc-baf4-1698be96568e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:19.304061 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7da425-0031-4abc-baf4-1698be96568e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:19.307156 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 53/105] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:07:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 818 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:19.314620 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7da425-0031-4abc-baf4-1698be96568e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:19.414190 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2825728-5fec-49d4-9b41-d1bd1dd104a0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:19.435721 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:19.459540 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:19.508176 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8204c6d-74ed-47a7-a456-09f0a400e1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:19.508176 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8204c6d-74ed-47a7-a456-09f0a400e1c1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:19.508176 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8204c6d-74ed-47a7-a456-09f0a400e1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:19.508176 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8204c6d-74ed-47a7-a456-09f0a400e1c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:19.606909 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7da425-0031-4abc-baf4-1698be96568e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:19.728020 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8204c6d-74ed-47a7-a456-09f0a400e1c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:19.729606 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8204c6d-74ed-47a7-a456-09f0a400e1c1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:19.734223 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186826fc-995d-4f6b-bf54-69ee4ebde5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:19.735312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186826fc-995d-4f6b-bf54-69ee4ebde5e0 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:19.735671 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186826fc-995d-4f6b-bf54-69ee4ebde5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:19.735972 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186826fc-995d-4f6b-bf54-69ee4ebde5e0 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:19.736813 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186826fc-995d-4f6b-bf54-69ee4ebde5e0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:19.738494 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186826fc-995d-4f6b-bf54-69ee4ebde5e0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:19.803657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 54/106] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Jul 25 19:07:19 2022] POST /identity/v3/projects => generated 302 bytes in 486 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:07:19.806410 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb8eea-2ed1-4bc4-9cae-dd37e4610cfd None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:19.806895 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb8eea-2ed1-4bc4-9cae-dd37e4610cfd None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:19.807354 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb8eea-2ed1-4bc4-9cae-dd37e4610cfd None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:19.811564 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 55/107] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:07:19.817333 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2296edd-6d17-4c6c-a3ce-307d4a262df2 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:19.817840 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2296edd-6d17-4c6c-a3ce-307d4a262df2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:19.820241 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2296edd-6d17-4c6c-a3ce-307d4a262df2 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:19.834803 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 56/108] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:19 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:07:19.837173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868ecff-a9d0-4fa0-ab1f-89af3f7833db None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:19.837622 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868ecff-a9d0-4fa0-ab1f-89af3f7833db None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:19.838078 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868ecff-a9d0-4fa0-ab1f-89af3f7833db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:19.903305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e868ecff-a9d0-4fa0-ab1f-89af3f7833db None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:19.907892 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e868ecff-a9d0-4fa0-ab1f-89af3f7833db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:20.011281 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:20.020944 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 53/109] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:07:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 893 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 19:07:20.023861 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178ca86-0f2a-460f-8228-964d1245b039 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:20.028693 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178ca86-0f2a-460f-8228-964d1245b039 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:20.029362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e178ca86-0f2a-460f-8228-964d1245b039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:20.066286 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e178ca86-0f2a-460f-8228-964d1245b039 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:20.069070 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e178ca86-0f2a-460f-8228-964d1245b039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:20.103570 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 57/110] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:07:20.105134 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e868ecff-a9d0-4fa0-ab1f-89af3f7833db None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:20.189925 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:20.232298 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 54/111] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:07:20.234601 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e178ca86-0f2a-460f-8228-964d1245b039 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:20.432339 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e868ecff-a9d0-4fa0-ab1f-89af3f7833db None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:20.459407 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e178ca86-0f2a-460f-8228-964d1245b039 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:20.489303 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade6d5d-788e-4c25-b609-11b893f5c19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:20.490499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade6d5d-788e-4c25-b609-11b893f5c19e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:20.490839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade6d5d-788e-4c25-b609-11b893f5c19e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:20.491118 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade6d5d-788e-4c25-b609-11b893f5c19e None admin] PATH_INFO: `/v3/projects/09f6174e875f4cf58a2eb5019ef11189` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:20.493355 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade6d5d-788e-4c25-b609-11b893f5c19e None admin] RBAC: Authorizing `identity:get_project(project_id=09f6174e875f4cf58a2eb5019ef11189)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:20.496172 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade6d5d-788e-4c25-b609-11b893f5c19e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:20.498632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 58/112] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:07:20 2022] GET /identity/v3/projects/09f6174e875f4cf58a2eb5019ef11189 => generated 298 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:20.506240 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebd931-d665-47d0-b67a-7b20f778235f None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:20.506737 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebd931-d665-47d0-b67a-7b20f778235f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:20.507246 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebd931-d665-47d0-b67a-7b20f778235f None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:20.509921 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 59/113] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:20 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:07:20.511370 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ebd931-d665-47d0-b67a-7b20f778235f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:20.537663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd05320-1414-4636-97e6-83521a6e86c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:20.540477 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd05320-1414-4636-97e6-83521a6e86c0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:20.540975 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd05320-1414-4636-97e6-83521a6e86c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:20.541483 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd05320-1414-4636-97e6-83521a6e86c0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:20.551814 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:20.569243 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd05320-1414-4636-97e6-83521a6e86c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:07:20.590341 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-0bd05320-1414-4636-97e6-83521a6e86c0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 19:07:20.590341 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:07:20.590341 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:07:20.590341 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:07:20.590341 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:07:20.590341 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:07:20.590341 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:07:20.590341 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:07:20.590341 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:07:20.590341 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:07:20.590341 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:07:20.590341 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:07:20.590341 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:07:20.590341 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:07:20.590341 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:07:20.590341 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:07:20.592267 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:07:20.592267 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:07:20.592267 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:07:20.592267 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:07:20.592267 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:07:20.592267 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:07:20.592267 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:07:20.592267 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:07:20.592267 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:07:20.592267 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:07:20.592267 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:07:20.592267 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:07:20.592267 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:07:20.592267 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:07:20.592267 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:07:20.592267 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:07:20.593191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:07:20.593191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:07:20.593191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:07:20.593191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:07:20.593191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:07:20.593191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:07:20.593191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:07:20.593191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:07:20.593191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:07:20.593191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:07:20.593191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 19:07:20.593191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:07:20.595961 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 55/114] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Jul 25 19:07:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:07:20.597492 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd05320-1414-4636-97e6-83521a6e86c0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:20.708659 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:20.731565 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ebd931-d665-47d0-b67a-7b20f778235f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:20.800210 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0cab4f-fcf9-43be-a7a5-ed79205e084b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:20.801529 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0cab4f-fcf9-43be-a7a5-ed79205e084b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:20.802020 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0cab4f-fcf9-43be-a7a5-ed79205e084b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:20.802506 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0cab4f-fcf9-43be-a7a5-ed79205e084b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:20.803969 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd05320-1414-4636-97e6-83521a6e86c0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:20.821909 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0cab4f-fcf9-43be-a7a5-ed79205e084b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:07:20.836389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-1b0cab4f-fcf9-43be-a7a5-ed79205e084b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 19:07:20.836389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:07:20.836389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:07:20.836389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:07:20.836389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:07:20.836389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:07:20.836389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:07:20.836389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:07:20.836389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:07:20.836389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:07:20.836389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:07:20.836389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:07:20.836389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:07:20.836389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:07:20.836389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:07:20.836389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:07:20.838063 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:07:20.838063 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:07:20.838063 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:07:20.838063 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:07:20.838063 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:07:20.838063 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:07:20.838063 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:07:20.838063 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:07:20.838063 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:07:20.838063 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:07:20.838063 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:07:20.838063 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:07:20.838063 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:07:20.838063 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:07:20.838063 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:07:20.838063 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:07:20.839779 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:07:20.839779 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:07:20.839779 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:07:20.839779 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:07:20.839779 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:07:20.839779 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:07:20.839779 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:07:20.839779 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:07:20.839779 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:07:20.839779 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:07:20.839779 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 19:07:20.839779 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:07:20.844707 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 60/115] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:07:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:07:20.847095 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca82119-6b36-4f63-b200-c797ebc02bd9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:20.847614 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca82119-6b36-4f63-b200-c797ebc02bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:20.848009 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca82119-6b36-4f63-b200-c797ebc02bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:20.869504 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69599f9-3de5-4d48-ac46-51b35fcd7e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:20.870569 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69599f9-3de5-4d48-ac46-51b35fcd7e55 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:20.871038 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69599f9-3de5-4d48-ac46-51b35fcd7e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:20.871500 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69599f9-3de5-4d48-ac46-51b35fcd7e55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:20.873046 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69599f9-3de5-4d48-ac46-51b35fcd7e55 None admin] RBAC: Adding query filter params (role.id=6327898c72b044889755dec9746f54dd, scope.project.id=09f6174e875f4cf58a2eb5019ef11189, user.id=7fc422bf660c4fd1b37fbcc1488b9173) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:20.873537 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69599f9-3de5-4d48-ac46-51b35fcd7e55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:20.876447 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69599f9-3de5-4d48-ac46-51b35fcd7e55 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:20.878753 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8ca82119-6b36-4f63-b200-c797ebc02bd9 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:20.880805 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8ca82119-6b36-4f63-b200-c797ebc02bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:20.889369 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 56/116] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:07:20 2022] GET /identity/v3/role_assignments?user.id=7fc422bf660c4fd1b37fbcc1488b9173&scope.project.id=09f6174e875f4cf58a2eb5019ef11189&role.id=6327898c72b044889755dec9746f54dd => generated 259 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:20.889369 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69599f9-3de5-4d48-ac46-51b35fcd7e55 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:20.997463 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:21.028333 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 61/117] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:07:21.036246 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503c72c-04dd-4c05-af42-e71e7cb8b59b None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:21.036246 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503c72c-04dd-4c05-af42-e71e7cb8b59b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:21.036246 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503c72c-04dd-4c05-af42-e71e7cb8b59b None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:21.037762 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 62/118] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:21 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:07:21.038791 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c503c72c-04dd-4c05-af42-e71e7cb8b59b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:21.095845 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69599f9-3de5-4d48-ac46-51b35fcd7e55 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:21.159887 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91fda74-9e76-4dee-9f4b-378fd4c90274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:21.163136 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91fda74-9e76-4dee-9f4b-378fd4c90274 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:21.164033 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91fda74-9e76-4dee-9f4b-378fd4c90274 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:21.164593 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91fda74-9e76-4dee-9f4b-378fd4c90274 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:21.165896 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91fda74-9e76-4dee-9f4b-378fd4c90274 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:21.166386 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91fda74-9e76-4dee-9f4b-378fd4c90274 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:21.168812 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91fda74-9e76-4dee-9f4b-378fd4c90274 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:21.188035 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 57/119] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:07:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:21.188823 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:21.190278 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879acb6-58d4-4a09-bf72-07d4a3f1edd1 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:21.190927 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879acb6-58d4-4a09-bf72-07d4a3f1edd1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:21.191291 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879acb6-58d4-4a09-bf72-07d4a3f1edd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:21.233123 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c503c72c-04dd-4c05-af42-e71e7cb8b59b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:21.237304 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d879acb6-58d4-4a09-bf72-07d4a3f1edd1 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:21.241058 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d879acb6-58d4-4a09-bf72-07d4a3f1edd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:21.303396 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307c8bc-985f-48f4-8119-fc62e51cc8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:21.304743 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307c8bc-985f-48f4-8119-fc62e51cc8fc None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:21.305082 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307c8bc-985f-48f4-8119-fc62e51cc8fc None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:21.305645 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307c8bc-985f-48f4-8119-fc62e51cc8fc None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:21.307568 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0307c8bc-985f-48f4-8119-fc62e51cc8fc None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:21.307766 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0307c8bc-985f-48f4-8119-fc62e51cc8fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:21.310367 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0307c8bc-985f-48f4-8119-fc62e51cc8fc None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:21.327177 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 63/120] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Jul 25 19:07:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:21.337579 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307c8bc-985f-48f4-8119-fc62e51cc8fc None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:21.399861 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 58/121] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:07:21.401681 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d879acb6-58d4-4a09-bf72-07d4a3f1edd1 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:21.512760 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:21.586455 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307c8bc-985f-48f4-8119-fc62e51cc8fc None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:21.644014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d879acb6-58d4-4a09-bf72-07d4a3f1edd1 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:21.671404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8377ef34-f886-426c-9a6e-c24bb5e57b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:21.674435 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377ef34-f886-426c-9a6e-c24bb5e57b16 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:21.675194 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377ef34-f886-426c-9a6e-c24bb5e57b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:21.675891 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377ef34-f886-426c-9a6e-c24bb5e57b16 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:21.705950 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8377ef34-f886-426c-9a6e-c24bb5e57b16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:07:21.714124 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e138e2-f25a-4966-bf12-5d896d1c12b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:21.715628 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e138e2-f25a-4966-bf12-5d896d1c12b9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:21.716121 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e138e2-f25a-4966-bf12-5d896d1c12b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:21.716860 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e138e2-f25a-4966-bf12-5d896d1c12b9 None admin] PATH_INFO: `/v3/users/1ee13b9555134b9cafac7f5f0a47204b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:21.719908 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e138e2-f25a-4966-bf12-5d896d1c12b9 None admin] RBAC: Authorizing `identity:get_user(user_id=1ee13b9555134b9cafac7f5f0a47204b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:21.722630 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e138e2-f25a-4966-bf12-5d896d1c12b9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:21.724360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-8377ef34-f886-426c-9a6e-c24bb5e57b16 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:07:21.724360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:07:21.724360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:07:21.724360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:07:21.724360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:07:21.724360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:07:21.724360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:07:21.724360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:07:21.724360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:07:21.724360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:07:21.724360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:07:21.724360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:07:21.724360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:07:21.724360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:07:21.724360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:07:21.724360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:07:21.726499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:07:21.726499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:07:21.726499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:07:21.726499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:07:21.726499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:07:21.726499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:07:21.726499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:07:21.726499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:07:21.726499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:07:21.726499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:07:21.726499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:07:21.726499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:07:21.726499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:07:21.726499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:07:21.726499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:07:21.726499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:07:21.728027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:07:21.728027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:07:21.728027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:07:21.728027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:07:21.728027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:07:21.728027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:07:21.728027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:07:21.728027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:07:21.728027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:07:21.728027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:07:21.728027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:07:21.728027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:07:21.729135 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 59/122] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:07:21 2022] GET /identity/v3/users/1ee13b9555134b9cafac7f5f0a47204b => generated 275 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:21.729135 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e138e2-f25a-4966-bf12-5d896d1c12b9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:21.732294 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 64/123] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Jul 25 19:07:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:07:21.734320 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8377ef34-f886-426c-9a6e-c24bb5e57b16 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:21.740915 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:21.940554 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e138e2-f25a-4966-bf12-5d896d1c12b9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:21.941332 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8377ef34-f886-426c-9a6e-c24bb5e57b16 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:21.970800 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:21.993421 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c1817a-cf08-4a85-ad44-3fc3b0226b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:21.994824 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1817a-cf08-4a85-ad44-3fc3b0226b9a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:21.995345 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1817a-cf08-4a85-ad44-3fc3b0226b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:21.995902 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1817a-cf08-4a85-ad44-3fc3b0226b9a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:22.010792 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c1817a-cf08-4a85-ad44-3fc3b0226b9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:07:22.014460 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce8c87-1697-467f-8947-031fac7d899c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:22.016681 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce8c87-1697-467f-8947-031fac7d899c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:22.017191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce8c87-1697-467f-8947-031fac7d899c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:22.017688 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce8c87-1697-467f-8947-031fac7d899c None admin] PATH_INFO: `/v3/users/1ee13b9555134b9cafac7f5f0a47204b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:22.020063 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce8c87-1697-467f-8947-031fac7d899c None admin] RBAC: Authorizing `identity:get_user(user_id=1ee13b9555134b9cafac7f5f0a47204b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:22.022136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e6c1817a-cf08-4a85-ad44-3fc3b0226b9a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 19:07:22.022136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:07:22.022136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:07:22.022136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:07:22.022136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:07:22.022136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:07:22.022136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:07:22.022136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:07:22.022136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:07:22.022136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:07:22.022136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:07:22.022136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:07:22.022136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:07:22.022136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:07:22.022136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:07:22.022136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:07:22.023005 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:07:22.023005 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:07:22.023005 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:07:22.023005 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:07:22.023005 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:07:22.023005 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:07:22.023005 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:07:22.023005 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:07:22.023005 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:07:22.023005 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:07:22.023005 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:07:22.023005 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:07:22.023005 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:07:22.023005 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:07:22.023005 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:07:22.023005 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:07:22.023872 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:07:22.023872 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:07:22.023872 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:07:22.023872 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:07:22.023872 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:07:22.023872 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:07:22.023872 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:07:22.023872 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:07:22.023872 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:07:22.023872 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:07:22.023872 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 19:07:22.023872 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:07:22.024502 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce8c87-1697-467f-8947-031fac7d899c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:22.026028 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 60/124] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:07:21 2022] GET /identity/v3/users/1ee13b9555134b9cafac7f5f0a47204b => generated 275 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:22.026775 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 65/125] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:07:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:07:22.027876 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c1817a-cf08-4a85-ad44-3fc3b0226b9a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:22.038115 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce8c87-1697-467f-8947-031fac7d899c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:22.127200 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:22.190089 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c1817a-cf08-4a85-ad44-3fc3b0226b9a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:22.257449 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1502ccad-a74d-4782-adeb-3638b1d575f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:22.259010 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502ccad-a74d-4782-adeb-3638b1d575f1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:22.259555 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502ccad-a74d-4782-adeb-3638b1d575f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:22.260016 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502ccad-a74d-4782-adeb-3638b1d575f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:22.261933 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1502ccad-a74d-4782-adeb-3638b1d575f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:22.263195 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1502ccad-a74d-4782-adeb-3638b1d575f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:22.266071 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1502ccad-a74d-4782-adeb-3638b1d575f1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:22.272179 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce8c87-1697-467f-8947-031fac7d899c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:22.290522 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 66/126] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:07:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:22.292571 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1502ccad-a74d-4782-adeb-3638b1d575f1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:22.325868 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f418eca-d883-42c6-b5a1-f6905fdafe8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:22.327413 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f418eca-d883-42c6-b5a1-f6905fdafe8e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:22.328202 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f418eca-d883-42c6-b5a1-f6905fdafe8e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:22.328868 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f418eca-d883-42c6-b5a1-f6905fdafe8e None admin] PATH_INFO: `/v3/projects/09f6174e875f4cf58a2eb5019ef11189` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:22.330588 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f418eca-d883-42c6-b5a1-f6905fdafe8e None admin] RBAC: Authorizing `identity:get_project(project_id=09f6174e875f4cf58a2eb5019ef11189)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:22.332501 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f418eca-d883-42c6-b5a1-f6905fdafe8e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:22.334913 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 61/127] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:07:22 2022] GET /identity/v3/projects/09f6174e875f4cf58a2eb5019ef11189 => generated 298 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:22.336149 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f418eca-d883-42c6-b5a1-f6905fdafe8e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:22.448758 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:22.485598 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1502ccad-a74d-4782-adeb-3638b1d575f1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:22.526524 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:22.544583 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382bcd2-0ed4-4b32-9eae-4dd2478c5d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:22.545731 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382bcd2-0ed4-4b32-9eae-4dd2478c5d62 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:22.546984 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382bcd2-0ed4-4b32-9eae-4dd2478c5d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:22.547837 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382bcd2-0ed4-4b32-9eae-4dd2478c5d62 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:22.549135 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3382bcd2-0ed4-4b32-9eae-4dd2478c5d62 None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:22.549893 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3382bcd2-0ed4-4b32-9eae-4dd2478c5d62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:22.552291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3382bcd2-0ed4-4b32-9eae-4dd2478c5d62 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:22.563693 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 67/128] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Jul 25 19:07:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:22.564999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382bcd2-0ed4-4b32-9eae-4dd2478c5d62 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:22.578544 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f418eca-d883-42c6-b5a1-f6905fdafe8e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:22.662229 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558dd692-abac-401b-b3a5-c6127d021fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:22.663809 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558dd692-abac-401b-b3a5-c6127d021fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:22.664370 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558dd692-abac-401b-b3a5-c6127d021fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:22.664805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558dd692-abac-401b-b3a5-c6127d021fa5 None admin] PATH_INFO: `/v3/projects/09f6174e875f4cf58a2eb5019ef11189` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:22.667639 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558dd692-abac-401b-b3a5-c6127d021fa5 None admin] RBAC: Authorizing `identity:get_project(project_id=09f6174e875f4cf58a2eb5019ef11189)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:22.670916 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558dd692-abac-401b-b3a5-c6127d021fa5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:22.679947 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 62/129] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:07:22 2022] GET /identity/v3/projects/09f6174e875f4cf58a2eb5019ef11189 => generated 298 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:22.681498 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558dd692-abac-401b-b3a5-c6127d021fa5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:22.732086 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382bcd2-0ed4-4b32-9eae-4dd2478c5d62 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:22.802414 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed7c355-5245-43b1-86f5-64381fe4f076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:22.804399 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7c355-5245-43b1-86f5-64381fe4f076 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:22.805334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7c355-5245-43b1-86f5-64381fe4f076 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:22.806087 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7c355-5245-43b1-86f5-64381fe4f076 None admin] PATH_INFO: `/v3/users/35d700f1abf943789efbf8a08bd9b587` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:22.808714 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed7c355-5245-43b1-86f5-64381fe4f076 None admin] RBAC: Authorizing `identity:get_user(user_id=35d700f1abf943789efbf8a08bd9b587)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:22.811369 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed7c355-5245-43b1-86f5-64381fe4f076 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:22.814671 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 68/130] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:07:22 2022] GET /identity/v3/users/35d700f1abf943789efbf8a08bd9b587 => generated 247 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:22.816030 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed7c355-5245-43b1-86f5-64381fe4f076 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:22.886937 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558dd692-abac-401b-b3a5-c6127d021fa5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:22.923769 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:22.933036 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:22.967479 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442d2852-1255-464b-a611-fe8af4fc97be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:22.969723 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d2852-1255-464b-a611-fe8af4fc97be None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:22.970166 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d2852-1255-464b-a611-fe8af4fc97be None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:22.970462 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d2852-1255-464b-a611-fe8af4fc97be None admin] PATH_INFO: `/v3/projects/09f6174e875f4cf58a2eb5019ef11189/users/1ee13b9555134b9cafac7f5f0a47204b/roles/191e7e31dad148b7a0eb52d3afb5e50e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:22.975783 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442d2852-1255-464b-a611-fe8af4fc97be None admin] RBAC: Authorizing `identity:create_grant(project_id=09f6174e875f4cf58a2eb5019ef11189, user_id=1ee13b9555134b9cafac7f5f0a47204b, role_id=191e7e31dad148b7a0eb52d3afb5e50e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:22.980060 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442d2852-1255-464b-a611-fe8af4fc97be None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:23.008463 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 63/131] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Jul 25 19:07:22 2022] PUT /identity/v3/projects/09f6174e875f4cf58a2eb5019ef11189/users/1ee13b9555134b9cafac7f5f0a47204b/roles/191e7e31dad148b7a0eb52d3afb5e50e => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:07:23.010028 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442d2852-1255-464b-a611-fe8af4fc97be None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:23.024403 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed7c355-5245-43b1-86f5-64381fe4f076 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:23.109588 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f78a9-17b1-4401-8a9f-74a51ebec318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:23.111529 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f78a9-17b1-4401-8a9f-74a51ebec318 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:23.112250 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f78a9-17b1-4401-8a9f-74a51ebec318 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:23.112884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f78a9-17b1-4401-8a9f-74a51ebec318 None admin] PATH_INFO: `/v3/users/1ee13b9555134b9cafac7f5f0a47204b` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:23.115218 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f78a9-17b1-4401-8a9f-74a51ebec318 None admin] RBAC: Authorizing `identity:get_user(user_id=1ee13b9555134b9cafac7f5f0a47204b)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:23.117768 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f78a9-17b1-4401-8a9f-74a51ebec318 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:23.120147 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 69/132] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:07:22 2022] GET /identity/v3/users/1ee13b9555134b9cafac7f5f0a47204b => generated 275 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:23.122172 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77ee41-526f-42fb-a19a-607ff9601abe None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:23.123691 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77ee41-526f-42fb-a19a-607ff9601abe None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:23.124797 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77ee41-526f-42fb-a19a-607ff9601abe None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:23.132486 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 70/133] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:07:23.133748 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77ee41-526f-42fb-a19a-607ff9601abe None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:23.210097 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442d2852-1255-464b-a611-fe8af4fc97be None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:23.280059 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c5a90-539e-4eee-8e2c-00c22181f54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:23.282678 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c5a90-539e-4eee-8e2c-00c22181f54c None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:23.283420 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c5a90-539e-4eee-8e2c-00c22181f54c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:23.284202 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c5a90-539e-4eee-8e2c-00c22181f54c None admin] PATH_INFO: `/v3/projects/09f6174e875f4cf58a2eb5019ef11189/users/1ee13b9555134b9cafac7f5f0a47204b/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:23.288812 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77c5a90-539e-4eee-8e2c-00c22181f54c None admin] RBAC: Authorizing `identity:create_grant(project_id=09f6174e875f4cf58a2eb5019ef11189, user_id=1ee13b9555134b9cafac7f5f0a47204b, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:23.291631 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77c5a90-539e-4eee-8e2c-00c22181f54c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:23.318729 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 64/134] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Jul 25 19:07:23 2022] PUT /identity/v3/projects/09f6174e875f4cf58a2eb5019ef11189/users/1ee13b9555134b9cafac7f5f0a47204b/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:07:23.320543 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c5a90-539e-4eee-8e2c-00c22181f54c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:23.368133 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77ee41-526f-42fb-a19a-607ff9601abe None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:23.403923 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:23.452137 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:23.485227 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369e331-af11-4044-8d89-b31db9968be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:23.486398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369e331-af11-4044-8d89-b31db9968be8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:23.487560 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369e331-af11-4044-8d89-b31db9968be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:23.489601 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369e331-af11-4044-8d89-b31db9968be8 None admin] PATH_INFO: `/v3/projects/09f6174e875f4cf58a2eb5019ef11189` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:23.492991 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369e331-af11-4044-8d89-b31db9968be8 None admin] RBAC: Authorizing `identity:get_project(project_id=09f6174e875f4cf58a2eb5019ef11189)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:23.499703 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369e331-af11-4044-8d89-b31db9968be8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:23.502215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 71/135] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:07:23 2022] GET /identity/v3/projects/09f6174e875f4cf58a2eb5019ef11189 => generated 298 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:23.504975 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe56662-5712-4b13-b5cd-bbe2d8521ff6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:23.508095 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe56662-5712-4b13-b5cd-bbe2d8521ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:23.509542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe56662-5712-4b13-b5cd-bbe2d8521ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:23.565340 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c5a90-539e-4eee-8e2c-00c22181f54c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:23.569725 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2fe56662-5712-4b13-b5cd-bbe2d8521ff6 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:23.572670 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2fe56662-5712-4b13-b5cd-bbe2d8521ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:23.617985 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93090743-8f9c-4f7b-adfd-434078fd85a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:23.619576 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93090743-8f9c-4f7b-adfd-434078fd85a4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:23.620013 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93090743-8f9c-4f7b-adfd-434078fd85a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:23.620296 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93090743-8f9c-4f7b-adfd-434078fd85a4 None admin] PATH_INFO: `/v3/projects/c1814fe30f064bc1875e3a1eac8ed490` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:23.622480 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93090743-8f9c-4f7b-adfd-434078fd85a4 None admin] RBAC: Authorizing `identity:get_project(project_id=c1814fe30f064bc1875e3a1eac8ed490)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:23.624312 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93090743-8f9c-4f7b-adfd-434078fd85a4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:23.628251 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 65/136] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:07:23 2022] GET /identity/v3/projects/c1814fe30f064bc1875e3a1eac8ed490 => generated 312 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:23.634013 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93090743-8f9c-4f7b-adfd-434078fd85a4 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:23.709822 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 72/137] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:07:23.713765 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe56662-5712-4b13-b5cd-bbe2d8521ff6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:23.910688 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93090743-8f9c-4f7b-adfd-434078fd85a4 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:23.915726 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:23.971946 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe56662-5712-4b13-b5cd-bbe2d8521ff6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:24.017581 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:24.056156 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e661d10-c74d-4a71-80c5-446ba78fdd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:24.058507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e661d10-c74d-4a71-80c5-446ba78fdd3b None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:24.059022 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e661d10-c74d-4a71-80c5-446ba78fdd3b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:24.059549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e661d10-c74d-4a71-80c5-446ba78fdd3b None admin] PATH_INFO: `/v3/projects/09f6174e875f4cf58a2eb5019ef11189/users/35d700f1abf943789efbf8a08bd9b587/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:24.064917 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e661d10-c74d-4a71-80c5-446ba78fdd3b None admin] RBAC: Authorizing `identity:create_grant(project_id=09f6174e875f4cf58a2eb5019ef11189, user_id=35d700f1abf943789efbf8a08bd9b587, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:24.067712 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e661d10-c74d-4a71-80c5-446ba78fdd3b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:24.070082 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1fe7e-6464-48ff-8a85-24a42bd51d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:24.072347 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1fe7e-6464-48ff-8a85-24a42bd51d69 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:24.073375 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1fe7e-6464-48ff-8a85-24a42bd51d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:24.073918 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1fe7e-6464-48ff-8a85-24a42bd51d69 None admin] PATH_INFO: `/v3/projects/c1814fe30f064bc1875e3a1eac8ed490/users/1ee13b9555134b9cafac7f5f0a47204b/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:24.088906 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1fe7e-6464-48ff-8a85-24a42bd51d69 None admin] RBAC: Authorizing `identity:create_grant(project_id=c1814fe30f064bc1875e3a1eac8ed490, user_id=1ee13b9555134b9cafac7f5f0a47204b, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:24.098218 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1fe7e-6464-48ff-8a85-24a42bd51d69 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:24.108913 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 66/138] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Jul 25 19:07:23 2022] PUT /identity/v3/projects/09f6174e875f4cf58a2eb5019ef11189/users/35d700f1abf943789efbf8a08bd9b587/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:07:24.111297 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e661d10-c74d-4a71-80c5-446ba78fdd3b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:24.141767 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 73/139] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Jul 25 19:07:23 2022] PUT /identity/v3/projects/c1814fe30f064bc1875e3a1eac8ed490/users/1ee13b9555134b9cafac7f5f0a47204b/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:07:24.351598 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e661d10-c74d-4a71-80c5-446ba78fdd3b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:24.367985 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f01b4-6081-4ec3-8048-4afcaefff9e0 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:24.368544 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f01b4-6081-4ec3-8048-4afcaefff9e0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:24.369095 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f01b4-6081-4ec3-8048-4afcaefff9e0 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:24.371830 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 74/140] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:07:24.383216 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d87fa-acd8-46a0-ba5e-c7be886818f6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:24.387033 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d87fa-acd8-46a0-ba5e-c7be886818f6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:24.387715 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d87fa-acd8-46a0-ba5e-c7be886818f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:24.453982 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2b8d87fa-acd8-46a0-ba5e-c7be886818f6 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:24.456746 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2b8d87fa-acd8-46a0-ba5e-c7be886818f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:24.520822 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9c384-7c64-4cc2-a2be-c5e66e8861f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:24.526773 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9c384-7c64-4cc2-a2be-c5e66e8861f9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:24.527829 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9c384-7c64-4cc2-a2be-c5e66e8861f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:24.528669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9c384-7c64-4cc2-a2be-c5e66e8861f9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:24.534682 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec9c384-7c64-4cc2-a2be-c5e66e8861f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:24.538222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec9c384-7c64-4cc2-a2be-c5e66e8861f9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:24.546094 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 67/141] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:07:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 437 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:24.557174 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9c384-7c64-4cc2-a2be-c5e66e8861f9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:24.669669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:24.705337 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 75/142] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 19:07:24.783925 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d87fa-acd8-46a0-ba5e-c7be886818f6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:24.839546 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9c384-7c64-4cc2-a2be-c5e66e8861f9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:24.943559 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d50c58-3363-4929-bad2-16c420acacc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:24.943559 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d50c58-3363-4929-bad2-16c420acacc0 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:24.943559 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d50c58-3363-4929-bad2-16c420acacc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:24.943559 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d50c58-3363-4929-bad2-16c420acacc0 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:24.944884 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d50c58-3363-4929-bad2-16c420acacc0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:24.944884 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d50c58-3363-4929-bad2-16c420acacc0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:25.030004 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 68/143] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Jul 25 19:07:24 2022] POST /identity/v3/users => generated 283 bytes in 474 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:07:25.130401 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d87fa-acd8-46a0-ba5e-c7be886818f6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:25.157791 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:25.241869 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa82525-e3af-446c-8d84-e1210b9ef4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:25.244029 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa82525-e3af-446c-8d84-e1210b9ef4bd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:25.245190 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa82525-e3af-446c-8d84-e1210b9ef4bd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:25.246293 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa82525-e3af-446c-8d84-e1210b9ef4bd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:25.278109 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa82525-e3af-446c-8d84-e1210b9ef4bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:07:25.301191 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-afa82525-e3af-446c-8d84-e1210b9ef4bd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 19:07:25.301191 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:07:25.301191 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:07:25.301191 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:07:25.301191 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:07:25.301191 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:07:25.301191 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:07:25.301191 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:07:25.301191 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:07:25.301191 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:07:25.301191 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:07:25.301191 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:07:25.301191 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:07:25.301191 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:07:25.301191 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:07:25.301191 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:07:25.302822 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:07:25.302822 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:07:25.302822 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:07:25.302822 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:07:25.302822 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:07:25.302822 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:07:25.302822 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:07:25.302822 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:07:25.302822 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:07:25.302822 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:07:25.302822 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:07:25.302822 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:07:25.302822 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:07:25.302822 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:07:25.302822 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:07:25.302822 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:07:25.309146 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:07:25.309146 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:07:25.309146 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:07:25.309146 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:07:25.309146 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:07:25.309146 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:07:25.309146 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:07:25.309146 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:07:25.309146 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:07:25.309146 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:07:25.309146 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 19:07:25.309146 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:07:25.314698 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 76/144] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:07:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 533 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:07:25.325787 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d50c58-3363-4929-bad2-16c420acacc0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:25.572675 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:25.600760 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d50c58-3363-4929-bad2-16c420acacc0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:25.681322 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff586cf-046b-4e64-aa0b-2c16dbc53051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:25.682569 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff586cf-046b-4e64-aa0b-2c16dbc53051 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:25.683087 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff586cf-046b-4e64-aa0b-2c16dbc53051 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:25.683638 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff586cf-046b-4e64-aa0b-2c16dbc53051 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:25.685191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff586cf-046b-4e64-aa0b-2c16dbc53051 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:25.685776 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff586cf-046b-4e64-aa0b-2c16dbc53051 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:25.689243 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff586cf-046b-4e64-aa0b-2c16dbc53051 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:25.716992 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 69/145] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Jul 25 19:07:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:25.736538 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa82525-e3af-446c-8d84-e1210b9ef4bd None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:25.964183 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa82525-e3af-446c-8d84-e1210b9ef4bd None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:26.046670 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e77f8-74ae-451c-8136-103ef4c6322c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:26.052725 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e77f8-74ae-451c-8136-103ef4c6322c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:26.053258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e77f8-74ae-451c-8136-103ef4c6322c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:26.053784 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e77f8-74ae-451c-8136-103ef4c6322c None admin] PATH_INFO: `/v3/users/7fc422bf660c4fd1b37fbcc1488b9173` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:26.056414 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1e77f8-74ae-451c-8136-103ef4c6322c None admin] RBAC: Authorizing `identity:get_user(user_id=7fc422bf660c4fd1b37fbcc1488b9173)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:26.059296 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1e77f8-74ae-451c-8136-103ef4c6322c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:26.062724 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 77/146] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:07:25 2022] GET /identity/v3/users/7fc422bf660c4fd1b37fbcc1488b9173 => generated 289 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:26.072119 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff586cf-046b-4e64-aa0b-2c16dbc53051 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:26.317916 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff586cf-046b-4e64-aa0b-2c16dbc53051 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:26.404043 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b123d-a98a-4f76-b27f-a5fb5696852d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:26.406250 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b123d-a98a-4f76-b27f-a5fb5696852d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:26.407087 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b123d-a98a-4f76-b27f-a5fb5696852d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:26.407196 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b123d-a98a-4f76-b27f-a5fb5696852d None admin] PATH_INFO: `/v3/projects/09f6174e875f4cf58a2eb5019ef11189` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:26.413494 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b123d-a98a-4f76-b27f-a5fb5696852d None admin] RBAC: Authorizing `identity:get_project(project_id=09f6174e875f4cf58a2eb5019ef11189)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:26.416401 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b123d-a98a-4f76-b27f-a5fb5696852d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:26.419862 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 70/147] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:07:26 2022] GET /identity/v3/projects/09f6174e875f4cf58a2eb5019ef11189 => generated 298 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:07:26.429168 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e77f8-74ae-451c-8136-103ef4c6322c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:26.531049 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:26.689877 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e77f8-74ae-451c-8136-103ef4c6322c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:26.781190 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fe82e7-abcf-4575-8d4f-912d92564673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:26.782831 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe82e7-abcf-4575-8d4f-912d92564673 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:26.783342 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe82e7-abcf-4575-8d4f-912d92564673 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:26.784419 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe82e7-abcf-4575-8d4f-912d92564673 None admin] PATH_INFO: `/v3/projects/09f6174e875f4cf58a2eb5019ef11189/users/7fc422bf660c4fd1b37fbcc1488b9173/roles/6327898c72b044889755dec9746f54dd` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:26.788866 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fe82e7-abcf-4575-8d4f-912d92564673 None admin] RBAC: Authorizing `identity:create_grant(project_id=09f6174e875f4cf58a2eb5019ef11189, user_id=7fc422bf660c4fd1b37fbcc1488b9173, role_id=6327898c72b044889755dec9746f54dd)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:26.791993 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fe82e7-abcf-4575-8d4f-912d92564673 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:26.819491 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 78/148] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Jul 25 19:07:26 2022] PUT /identity/v3/projects/09f6174e875f4cf58a2eb5019ef11189/users/7fc422bf660c4fd1b37fbcc1488b9173/roles/6327898c72b044889755dec9746f54dd => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:07:27.053392 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483035a-d38e-496a-b895-4851326b7c39 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:27.053787 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483035a-d38e-496a-b895-4851326b7c39 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:27.054314 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483035a-d38e-496a-b895-4851326b7c39 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:27.057143 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 71/149] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:07:27.081664 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9400c28-2f6b-4119-9c22-df3b4d9232d5 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:27.082157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9400c28-2f6b-4119-9c22-df3b4d9232d5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:27.082679 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9400c28-2f6b-4119-9c22-df3b4d9232d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:27.124396 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e9400c28-2f6b-4119-9c22-df3b4d9232d5 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:27.126285 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e9400c28-2f6b-4119-9c22-df3b4d9232d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:27.267962 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 79/150] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 25 19:07:27.326524 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d483035a-d38e-496a-b895-4851326b7c39 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:27.366713 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:27.518166 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d483035a-d38e-496a-b895-4851326b7c39 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:27.574032 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74980ce-fde7-4f9b-b3c8-ecd21c8a70c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:27.574992 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74980ce-fde7-4f9b-b3c8-ecd21c8a70c5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:27.575295 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74980ce-fde7-4f9b-b3c8-ecd21c8a70c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:27.575609 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74980ce-fde7-4f9b-b3c8-ecd21c8a70c5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:27.593091 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74980ce-fde7-4f9b-b3c8-ecd21c8a70c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:07:27.603891 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-e74980ce-fde7-4f9b-b3c8-ecd21c8a70c5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 19:07:27.603891 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:07:27.603891 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:07:27.603891 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:07:27.603891 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:07:27.603891 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:07:27.603891 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:07:27.603891 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:07:27.603891 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:07:27.603891 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:07:27.603891 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:07:27.603891 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:07:27.603891 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:07:27.603891 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:07:27.603891 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:07:27.603891 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:07:27.605489 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:07:27.605489 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:07:27.605489 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:07:27.605489 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:07:27.605489 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:07:27.605489 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:07:27.605489 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:07:27.605489 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:07:27.605489 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:07:27.605489 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:07:27.605489 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:07:27.605489 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:07:27.605489 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:07:27.605489 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:07:27.605489 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:07:27.605489 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:07:27.608678 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:07:27.608678 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:07:27.608678 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:07:27.608678 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:07:27.608678 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:07:27.608678 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:07:27.608678 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:07:27.608678 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:07:27.608678 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:07:27.608678 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:07:27.608678 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 19:07:27.608678 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:07:27.613820 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 72/151] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Jul 25 19:07:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:07:27.617676 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9400c28-2f6b-4119-9c22-df3b4d9232d5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:27.699726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:27.848341 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9400c28-2f6b-4119-9c22-df3b4d9232d5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:27.927129 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3caefd3-fc4c-4593-8f73-e7674c437ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:27.927851 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3caefd3-fc4c-4593-8f73-e7674c437ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:27.928349 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3caefd3-fc4c-4593-8f73-e7674c437ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:27.928850 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3caefd3-fc4c-4593-8f73-e7674c437ee1 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:27.930397 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3caefd3-fc4c-4593-8f73-e7674c437ee1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:27.930957 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3caefd3-fc4c-4593-8f73-e7674c437ee1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:27.933688 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3caefd3-fc4c-4593-8f73-e7674c437ee1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:27.955953 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 80/152] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:07:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:27.962478 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74980ce-fde7-4f9b-b3c8-ecd21c8a70c5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:28.115825 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:28.147586 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74980ce-fde7-4f9b-b3c8-ecd21c8a70c5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:28.224959 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0a885-e81d-4fe6-a50a-54924a2bfc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:28.226649 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0a885-e81d-4fe6-a50a-54924a2bfc21 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:28.227160 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0a885-e81d-4fe6-a50a-54924a2bfc21 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:28.227734 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0a885-e81d-4fe6-a50a-54924a2bfc21 None admin] PATH_INFO: `/v3/users/1ee13b9555134b9cafac7f5f0a47204b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:28.230328 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb0a885-e81d-4fe6-a50a-54924a2bfc21 None admin] RBAC: Authorizing `identity:get_user(user_id=1ee13b9555134b9cafac7f5f0a47204b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:28.233373 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb0a885-e81d-4fe6-a50a-54924a2bfc21 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:28.237679 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 73/153] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:07:27 2022] GET /identity/v3/users/1ee13b9555134b9cafac7f5f0a47204b => generated 275 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:28.243212 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3caefd3-fc4c-4593-8f73-e7674c437ee1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:28.444036 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:28.471618 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3caefd3-fc4c-4593-8f73-e7674c437ee1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:28.553583 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f98e3c-6c04-44f9-9dbd-639f5b9afe1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:28.554272 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f98e3c-6c04-44f9-9dbd-639f5b9afe1d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:28.554561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f98e3c-6c04-44f9-9dbd-639f5b9afe1d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:28.554846 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f98e3c-6c04-44f9-9dbd-639f5b9afe1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:28.692476 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f98e3c-6c04-44f9-9dbd-639f5b9afe1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:28.694563 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f98e3c-6c04-44f9-9dbd-639f5b9afe1d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:28.852794 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:28.867925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2410a-14b4-4209-a8b4-4d77107b49c8 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:28.869370 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2410a-14b4-4209-a8b4-4d77107b49c8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:28.870072 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2410a-14b4-4209-a8b4-4d77107b49c8 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:28.872460 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 74/154] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:07:28.880097 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 81/155] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:07:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 638 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 19:07:28.882157 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b0d67-a690-4326-9391-e1a39fc8a2e9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:28.882871 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b0d67-a690-4326-9391-e1a39fc8a2e9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:28.883360 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b0d67-a690-4326-9391-e1a39fc8a2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:28.891544 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f98e3c-6c04-44f9-9dbd-639f5b9afe1d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:28.940795 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-907b0d67-a690-4326-9391-e1a39fc8a2e9 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:28.942463 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-907b0d67-a690-4326-9391-e1a39fc8a2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:29.187803 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 75/156] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 19:07:29.259727 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b0d67-a690-4326-9391-e1a39fc8a2e9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:29.272126 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f98e3c-6c04-44f9-9dbd-639f5b9afe1d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:29.416080 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:29.453404 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b0d67-a690-4326-9391-e1a39fc8a2e9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:29.481868 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752af499-043d-4852-b208-58f9597e6c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:29.484840 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752af499-043d-4852-b208-58f9597e6c6c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:29.485933 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752af499-043d-4852-b208-58f9597e6c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:29.486746 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752af499-043d-4852-b208-58f9597e6c6c None admin] PATH_INFO: `/v3/projects/09f6174e875f4cf58a2eb5019ef11189` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:29.490267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752af499-043d-4852-b208-58f9597e6c6c None admin] RBAC: Authorizing `identity:get_project(project_id=09f6174e875f4cf58a2eb5019ef11189)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:29.498017 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752af499-043d-4852-b208-58f9597e6c6c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:29.506874 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 82/157] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:07:28 2022] GET /identity/v3/projects/09f6174e875f4cf58a2eb5019ef11189 => generated 298 bytes in 619 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:29.517378 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752af499-043d-4852-b208-58f9597e6c6c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:29.521144 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a572f1b7-9b49-4194-bc7c-4a8964d13c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:29.522159 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572f1b7-9b49-4194-bc7c-4a8964d13c60 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:29.522497 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572f1b7-9b49-4194-bc7c-4a8964d13c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:29.522977 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572f1b7-9b49-4194-bc7c-4a8964d13c60 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:29.543556 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a572f1b7-9b49-4194-bc7c-4a8964d13c60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:07:29.555084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-a572f1b7-9b49-4194-bc7c-4a8964d13c60 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 19:07:29.555084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:07:29.555084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:07:29.555084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:07:29.555084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:07:29.555084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:07:29.555084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:07:29.555084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:07:29.555084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:07:29.555084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:07:29.555084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:07:29.555084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:07:29.555084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:07:29.555084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:07:29.555084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:07:29.555084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:07:29.560511 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:07:29.560511 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:07:29.560511 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:07:29.560511 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:07:29.560511 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:07:29.560511 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:07:29.560511 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:07:29.560511 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:07:29.560511 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:07:29.560511 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:07:29.560511 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:07:29.560511 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:07:29.560511 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:07:29.560511 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:07:29.560511 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:07:29.560511 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:07:29.561661 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:07:29.561661 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:07:29.561661 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:07:29.561661 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:07:29.561661 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:07:29.561661 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:07:29.561661 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:07:29.561661 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:07:29.561661 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:07:29.561661 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:07:29.561661 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 19:07:29.561661 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:07:29.563320 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 76/158] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:07:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:07:29.571272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a572f1b7-9b49-4194-bc7c-4a8964d13c60 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:29.655803 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:29.714366 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752af499-043d-4852-b208-58f9597e6c6c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:29.788352 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c265308-8c28-4051-8550-e8577e4e1926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:29.789476 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c265308-8c28-4051-8550-e8577e4e1926 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:29.789974 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c265308-8c28-4051-8550-e8577e4e1926 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:29.790459 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c265308-8c28-4051-8550-e8577e4e1926 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:29.791634 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c265308-8c28-4051-8550-e8577e4e1926 None admin] RBAC: Adding query filter params (role.id=191e7e31dad148b7a0eb52d3afb5e50e, scope.project.id=09f6174e875f4cf58a2eb5019ef11189, user.id=1ee13b9555134b9cafac7f5f0a47204b) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:29.792247 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c265308-8c28-4051-8550-e8577e4e1926 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:29.794015 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c265308-8c28-4051-8550-e8577e4e1926 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:29.802163 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 83/159] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:07:29 2022] GET /identity/v3/role_assignments?user.id=1ee13b9555134b9cafac7f5f0a47204b&scope.project.id=09f6174e875f4cf58a2eb5019ef11189&role.id=191e7e31dad148b7a0eb52d3afb5e50e => generated 608 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:29.917966 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a572f1b7-9b49-4194-bc7c-4a8964d13c60 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:29.993004 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a289d-1378-43c6-bd4d-f843cf6a7487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:29.994090 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a289d-1378-43c6-bd4d-f843cf6a7487 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:29.994537 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a289d-1378-43c6-bd4d-f843cf6a7487 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:29.995002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a289d-1378-43c6-bd4d-f843cf6a7487 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:29.996454 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a289d-1378-43c6-bd4d-f843cf6a7487 None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:29.997010 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a289d-1378-43c6-bd4d-f843cf6a7487 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:30.000282 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a289d-1378-43c6-bd4d-f843cf6a7487 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:30.004878 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:30.021565 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 77/160] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Jul 25 19:07:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 451 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:30.032809 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c265308-8c28-4051-8550-e8577e4e1926 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:30.181568 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c265308-8c28-4051-8550-e8577e4e1926 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:30.215993 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:30.239802 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51756aa7-1097-43c8-b73f-1b1652abc014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:30.240928 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51756aa7-1097-43c8-b73f-1b1652abc014 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:30.241217 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51756aa7-1097-43c8-b73f-1b1652abc014 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:30.241499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51756aa7-1097-43c8-b73f-1b1652abc014 None admin] PATH_INFO: `/v3/users/1ee13b9555134b9cafac7f5f0a47204b` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:30.248915 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51756aa7-1097-43c8-b73f-1b1652abc014 None admin] RBAC: Authorizing `identity:get_user(user_id=1ee13b9555134b9cafac7f5f0a47204b)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:30.251443 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51756aa7-1097-43c8-b73f-1b1652abc014 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:30.253920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 84/161] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:07:30 2022] GET /identity/v3/users/1ee13b9555134b9cafac7f5f0a47204b => generated 275 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:30.265468 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a289d-1378-43c6-bd4d-f843cf6a7487 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:30.525632 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a289d-1378-43c6-bd4d-f843cf6a7487 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:30.598449 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc773e51-1c60-449f-8d41-2fd6c409815e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:30.599174 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc773e51-1c60-449f-8d41-2fd6c409815e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:30.599467 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc773e51-1c60-449f-8d41-2fd6c409815e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:30.599789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc773e51-1c60-449f-8d41-2fd6c409815e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:30.692935 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8643e-4205-41d7-af1a-d675194b8f60 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:30.693361 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8643e-4205-41d7-af1a-d675194b8f60 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:30.693482 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8643e-4205-41d7-af1a-d675194b8f60 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:30.694993 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 85/162] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:07:30.704349 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34e744-84f4-43d3-949e-0f596aa6e2c2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:30.704686 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34e744-84f4-43d3-949e-0f596aa6e2c2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:30.704916 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34e744-84f4-43d3-949e-0f596aa6e2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:30.730432 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:30.739650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc773e51-1c60-449f-8d41-2fd6c409815e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:30.741504 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc773e51-1c60-449f-8d41-2fd6c409815e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:30.750728 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ec34e744-84f4-43d3-949e-0f596aa6e2c2 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:30.752891 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ec34e744-84f4-43d3-949e-0f596aa6e2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:30.971334 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 78/163] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:07:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 707 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 19:07:30.979578 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471e8af-2da6-41f9-829f-c8642715bb04 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:30.979578 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471e8af-2da6-41f9-829f-c8642715bb04 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:30.979578 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471e8af-2da6-41f9-829f-c8642715bb04 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:30.979578 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 79/164] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:07:30.991540 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a471e8af-2da6-41f9-829f-c8642715bb04 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:31.002494 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 86/165] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 19:07:31.005791 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46f87b-6a9e-455e-b1b0-403a62e7d4f4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:31.006015 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46f87b-6a9e-455e-b1b0-403a62e7d4f4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:31.006388 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46f87b-6a9e-455e-b1b0-403a62e7d4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:31.065586 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-eb46f87b-6a9e-455e-b1b0-403a62e7d4f4 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:31.067776 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-eb46f87b-6a9e-455e-b1b0-403a62e7d4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:31.228455 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:31.245105 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a471e8af-2da6-41f9-829f-c8642715bb04 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:31.286889 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 87/166] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:07:31.287935 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb46f87b-6a9e-455e-b1b0-403a62e7d4f4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:31.364861 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aae48f-cd7b-4c79-9f7b-bd93f493c790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:31.366481 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aae48f-cd7b-4c79-9f7b-bd93f493c790 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:31.366986 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aae48f-cd7b-4c79-9f7b-bd93f493c790 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:31.367501 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aae48f-cd7b-4c79-9f7b-bd93f493c790 None admin] PATH_INFO: `/v3/projects/09f6174e875f4cf58a2eb5019ef11189` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:31.373002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aae48f-cd7b-4c79-9f7b-bd93f493c790 None admin] RBAC: Authorizing `identity:get_project(project_id=09f6174e875f4cf58a2eb5019ef11189)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:31.376355 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aae48f-cd7b-4c79-9f7b-bd93f493c790 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:31.379788 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 80/167] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:07:30 2022] GET /identity/v3/projects/09f6174e875f4cf58a2eb5019ef11189 => generated 298 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:07:31.386949 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aae48f-cd7b-4c79-9f7b-bd93f493c790 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:31.454195 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:31.646335 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb46f87b-6a9e-455e-b1b0-403a62e7d4f4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:31.777214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aae48f-cd7b-4c79-9f7b-bd93f493c790 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:31.800874 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237e234-9728-444d-a867-491e530b0268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:31.802208 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237e234-9728-444d-a867-491e530b0268 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:31.802703 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237e234-9728-444d-a867-491e530b0268 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:31.803203 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237e234-9728-444d-a867-491e530b0268 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:31.835886 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6237e234-9728-444d-a867-491e530b0268 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:07:31.855865 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-6237e234-9728-444d-a867-491e530b0268 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:07:31.855865 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:07:31.855865 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:07:31.855865 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:07:31.855865 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:07:31.855865 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:07:31.855865 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:07:31.855865 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:07:31.855865 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:07:31.855865 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:07:31.855865 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:07:31.855865 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:07:31.855865 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:07:31.855865 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:07:31.855865 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:07:31.855865 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:07:31.857972 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:07:31.857972 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:07:31.857972 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:07:31.857972 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:07:31.857972 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:07:31.857972 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:07:31.857972 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:07:31.857972 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:07:31.857972 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:07:31.857972 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:07:31.857972 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:07:31.857972 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:07:31.857972 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:07:31.857972 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:07:31.857972 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:07:31.857972 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:07:31.860465 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:07:31.860465 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:07:31.860465 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:07:31.860465 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:07:31.860465 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:07:31.860465 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:07:31.860465 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:07:31.860465 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:07:31.860465 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:07:31.860465 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:07:31.860465 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:07:31.860465 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:07:31.870004 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 88/168] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Jul 25 19:07:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 581 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:07:31.870004 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237e234-9728-444d-a867-491e530b0268 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:31.899863 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6b995-b17a-4c78-8cb3-71a56da1b562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:31.901308 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6b995-b17a-4c78-8cb3-71a56da1b562 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:31.901829 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6b995-b17a-4c78-8cb3-71a56da1b562 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:31.902346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6b995-b17a-4c78-8cb3-71a56da1b562 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:31.927548 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a6b995-b17a-4c78-8cb3-71a56da1b562 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:07:31.943779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-c1a6b995-b17a-4c78-8cb3-71a56da1b562 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 19:07:31.943779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:07:31.943779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:07:31.943779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:07:31.943779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:07:31.943779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:07:31.943779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:07:31.943779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:07:31.943779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:07:31.943779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:07:31.943779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:07:31.943779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:07:31.943779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:07:31.943779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:07:31.943779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:07:31.943779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:07:31.945576 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:07:31.945576 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:07:31.945576 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:07:31.945576 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:07:31.945576 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:07:31.945576 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:07:31.945576 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:07:31.945576 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:07:31.945576 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:07:31.945576 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:07:31.945576 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:07:31.945576 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:07:31.945576 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:07:31.945576 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:07:31.945576 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:07:31.945576 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:07:31.949680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:07:31.949680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:07:31.949680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:07:31.949680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:07:31.949680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:07:31.949680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:07:31.949680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:07:31.949680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:07:31.949680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:07:31.949680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:07:31.949680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 19:07:31.949680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:07:31.949680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 81/169] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:07:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 561 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:07:31.949680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b566dc6-5e06-402e-b8cf-e11494925672 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:31.961247 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b566dc6-5e06-402e-b8cf-e11494925672 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:31.961247 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b566dc6-5e06-402e-b8cf-e11494925672 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:31.961247 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 82/170] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:07:31.961247 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b566dc6-5e06-402e-b8cf-e11494925672 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:32.024913 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:32.204346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b566dc6-5e06-402e-b8cf-e11494925672 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:32.326698 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97257dda-3420-47e4-bd33-927c756acd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:32.329161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97257dda-3420-47e4-bd33-927c756acd34 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:32.329536 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97257dda-3420-47e4-bd33-927c756acd34 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:32.329780 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97257dda-3420-47e4-bd33-927c756acd34 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:32.330613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97257dda-3420-47e4-bd33-927c756acd34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:32.335564 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97257dda-3420-47e4-bd33-927c756acd34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:32.345681 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97257dda-3420-47e4-bd33-927c756acd34 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:32.371898 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237e234-9728-444d-a867-491e530b0268 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:32.374466 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 83/171] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:07:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:32.376314 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ac779-1371-4f76-a3a0-a0c9c176c5d3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:32.376618 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ac779-1371-4f76-a3a0-a0c9c176c5d3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:32.379562 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ac779-1371-4f76-a3a0-a0c9c176c5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:32.401827 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:32.418109 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ee1ac779-1371-4f76-a3a0-a0c9c176c5d3 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:32.420124 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ee1ac779-1371-4f76-a3a0-a0c9c176c5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:32.446129 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f69f2-2b53-4bf5-be71-8420f8c84e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:32.446845 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f69f2-2b53-4bf5-be71-8420f8c84e92 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:32.447280 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f69f2-2b53-4bf5-be71-8420f8c84e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:32.447813 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f69f2-2b53-4bf5-be71-8420f8c84e92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:32.449456 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830f69f2-2b53-4bf5-be71-8420f8c84e92 None admin] RBAC: Adding query filter params (role.id=c241bd9634e24bcbb1854c7621d19f81, scope.project.id=09f6174e875f4cf58a2eb5019ef11189, user.id=1ee13b9555134b9cafac7f5f0a47204b) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:32.450029 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830f69f2-2b53-4bf5-be71-8420f8c84e92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:32.453629 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830f69f2-2b53-4bf5-be71-8420f8c84e92 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:32.467128 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 89/172] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:07:31 2022] GET /identity/v3/role_assignments?user.id=1ee13b9555134b9cafac7f5f0a47204b&scope.project.id=09f6174e875f4cf58a2eb5019ef11189&role.id=c241bd9634e24bcbb1854c7621d19f81 => generated 608 bytes in 598 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:32.469310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f69f2-2b53-4bf5-be71-8420f8c84e92 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:32.619483 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 84/173] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:07:32.622158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f328cb-175e-441f-876a-6d34d918308f None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:32.622695 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f328cb-175e-441f-876a-6d34d918308f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:32.623199 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f328cb-175e-441f-876a-6d34d918308f None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:32.627118 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 85/174] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:07:32.628708 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f328cb-175e-441f-876a-6d34d918308f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:32.687827 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:32.694432 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f69f2-2b53-4bf5-be71-8420f8c84e92 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:32.761331 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df3bc6-305c-40a5-809f-aa8cd8b8b2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:32.762034 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df3bc6-305c-40a5-809f-aa8cd8b8b2a7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:32.762325 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df3bc6-305c-40a5-809f-aa8cd8b8b2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:32.762589 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df3bc6-305c-40a5-809f-aa8cd8b8b2a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:32.763381 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58df3bc6-305c-40a5-809f-aa8cd8b8b2a7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:32.763714 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58df3bc6-305c-40a5-809f-aa8cd8b8b2a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:32.766370 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58df3bc6-305c-40a5-809f-aa8cd8b8b2a7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:32.777993 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 90/175] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Jul 25 19:07:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:32.781379 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c70c40-6853-40ec-ae87-f082b527a3b8 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:32.781379 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c70c40-6853-40ec-ae87-f082b527a3b8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:32.781379 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c70c40-6853-40ec-ae87-f082b527a3b8 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:32.781893 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 91/176] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:07:32.783135 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675eb0af-1493-4152-b667-246781ec654e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:32.783419 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675eb0af-1493-4152-b667-246781ec654e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:32.783819 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675eb0af-1493-4152-b667-246781ec654e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:32.809276 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-675eb0af-1493-4152-b667-246781ec654e None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:32.810709 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-675eb0af-1493-4152-b667-246781ec654e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:32.843749 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f328cb-175e-441f-876a-6d34d918308f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:32.867289 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:32.890149 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac69a3-26a4-45e9-97f8-97e20a2457ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:32.891629 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac69a3-26a4-45e9-97f8-97e20a2457ff None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:32.892094 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac69a3-26a4-45e9-97f8-97e20a2457ff None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:32.892591 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac69a3-26a4-45e9-97f8-97e20a2457ff None admin] PATH_INFO: `/v3/users/35d700f1abf943789efbf8a08bd9b587` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:32.894168 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac69a3-26a4-45e9-97f8-97e20a2457ff None admin] RBAC: Authorizing `identity:get_user(user_id=35d700f1abf943789efbf8a08bd9b587)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:32.895884 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac69a3-26a4-45e9-97f8-97e20a2457ff None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:32.897804 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 86/177] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:07:32 2022] GET /identity/v3/users/35d700f1abf943789efbf8a08bd9b587 => generated 247 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:32.898447 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac69a3-26a4-45e9-97f8-97e20a2457ff None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:32.955655 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 92/178] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:07:32.955655 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675eb0af-1493-4152-b667-246781ec654e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:33.053135 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:33.079315 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac69a3-26a4-45e9-97f8-97e20a2457ff None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:33.141269 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675eb0af-1493-4152-b667-246781ec654e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:33.147209 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652408f5-1018-48a1-b055-77864b1b16bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:33.148678 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652408f5-1018-48a1-b055-77864b1b16bf None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:33.149186 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652408f5-1018-48a1-b055-77864b1b16bf None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:33.149543 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652408f5-1018-48a1-b055-77864b1b16bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:33.165913 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652408f5-1018-48a1-b055-77864b1b16bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:07:33.176618 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-652408f5-1018-48a1-b055-77864b1b16bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:07:33.176618 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:07:33.176618 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:07:33.176618 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:07:33.176618 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:07:33.176618 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:07:33.176618 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:07:33.176618 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:07:33.176618 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:07:33.176618 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:07:33.176618 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:07:33.176618 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:07:33.176618 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:07:33.176618 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:07:33.176618 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:07:33.176618 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:07:33.178502 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:07:33.178502 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:07:33.178502 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:07:33.178502 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:07:33.178502 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:07:33.178502 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:07:33.178502 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:07:33.178502 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:07:33.178502 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:07:33.178502 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:07:33.178502 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:07:33.178502 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:07:33.178502 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:07:33.178502 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:07:33.178502 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:07:33.178502 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:07:33.180133 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:07:33.180133 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:07:33.180133 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:07:33.180133 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:07:33.180133 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:07:33.180133 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:07:33.180133 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:07:33.180133 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:07:33.180133 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:07:33.180133 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:07:33.180133 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:07:33.180133 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:07:33.180133 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 87/179] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Jul 25 19:07:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:07:33.180133 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8375bf5-41f7-4e94-9cb7-0959d7fbc631 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:33.181632 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8375bf5-41f7-4e94-9cb7-0959d7fbc631 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:33.181632 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8375bf5-41f7-4e94-9cb7-0959d7fbc631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:33.198216 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d3048f-ac5f-4887-9b0b-6d2901cc8dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:33.199243 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3048f-ac5f-4887-9b0b-6d2901cc8dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:33.199751 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3048f-ac5f-4887-9b0b-6d2901cc8dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:33.200133 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3048f-ac5f-4887-9b0b-6d2901cc8dc6 None admin] PATH_INFO: `/v3/users/1ee13b9555134b9cafac7f5f0a47204b` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:33.202949 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d3048f-ac5f-4887-9b0b-6d2901cc8dc6 None admin] RBAC: Authorizing `identity:get_user(user_id=1ee13b9555134b9cafac7f5f0a47204b)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:33.204449 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f8375bf5-41f7-4e94-9cb7-0959d7fbc631 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:33.205446 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d3048f-ac5f-4887-9b0b-6d2901cc8dc6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:33.207147 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f8375bf5-41f7-4e94-9cb7-0959d7fbc631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:33.207728 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 93/180] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:07:32 2022] GET /identity/v3/users/1ee13b9555134b9cafac7f5f0a47204b => generated 275 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:33.208901 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d3048f-ac5f-4887-9b0b-6d2901cc8dc6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:33.256466 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:33.320719 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 88/181] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:07:33.322562 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8375bf5-41f7-4e94-9cb7-0959d7fbc631 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:33.373265 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d3048f-ac5f-4887-9b0b-6d2901cc8dc6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:33.427649 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8bbc78-c03e-4518-af96-6a16b77fdbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:33.428730 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8bbc78-c03e-4518-af96-6a16b77fdbf4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:33.429203 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8bbc78-c03e-4518-af96-6a16b77fdbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:33.429678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8bbc78-c03e-4518-af96-6a16b77fdbf4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:33.463019 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:33.509845 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8375bf5-41f7-4e94-9cb7-0959d7fbc631 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:33.544660 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8bbc78-c03e-4518-af96-6a16b77fdbf4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:33.547182 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8bbc78-c03e-4518-af96-6a16b77fdbf4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:33.582677 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e882b5a-2b29-4fcd-80b3-3cb354aa465b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:33.584200 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e882b5a-2b29-4fcd-80b3-3cb354aa465b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:33.584696 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e882b5a-2b29-4fcd-80b3-3cb354aa465b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:33.585203 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e882b5a-2b29-4fcd-80b3-3cb354aa465b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:33.602938 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:33.606563 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e882b5a-2b29-4fcd-80b3-3cb354aa465b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:07:33.623586 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-8e882b5a-2b29-4fcd-80b3-3cb354aa465b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 19:07:33.623586 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:07:33.623586 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:07:33.623586 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:07:33.623586 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:07:33.623586 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:07:33.623586 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:07:33.623586 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:07:33.623586 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:07:33.623586 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:07:33.623586 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:07:33.623586 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:07:33.623586 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:07:33.623586 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:07:33.623586 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:07:33.623586 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:07:33.625782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:07:33.625782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:07:33.625782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:07:33.625782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:07:33.625782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:07:33.625782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:07:33.625782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:07:33.625782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:07:33.625782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:07:33.625782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:07:33.625782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:07:33.625782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:07:33.625782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:07:33.625782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:07:33.625782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:07:33.625782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:07:33.626790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:07:33.626790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:07:33.626790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:07:33.626790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:07:33.626790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:07:33.626790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:07:33.626790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:07:33.626790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:07:33.626790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:07:33.626790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:07:33.626790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 19:07:33.626790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:07:33.627766 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 89/182] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Jul 25 19:07:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:07:33.629545 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bb4a7-61b4-42f9-b847-d0db917a4f12 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:33.630054 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bb4a7-61b4-42f9-b847-d0db917a4f12 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:33.630547 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bb4a7-61b4-42f9-b847-d0db917a4f12 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:33.633241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 90/183] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:07:33.634714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4bb4a7-61b4-42f9-b847-d0db917a4f12 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:33.734886 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 94/184] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:07:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 19:07:33.735996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8bbc78-c03e-4518-af96-6a16b77fdbf4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:33.809245 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4bb4a7-61b4-42f9-b847-d0db917a4f12 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:33.854595 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d76f7-7718-4028-af30-1825f07dd9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:33.855608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d76f7-7718-4028-af30-1825f07dd9d3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:33.856072 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d76f7-7718-4028-af30-1825f07dd9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:33.856521 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d76f7-7718-4028-af30-1825f07dd9d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:33.857940 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d76f7-7718-4028-af30-1825f07dd9d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:33.859117 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d76f7-7718-4028-af30-1825f07dd9d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:33.861601 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d76f7-7718-4028-af30-1825f07dd9d3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:33.866026 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:33.878926 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 91/185] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:07:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:33.880337 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d76f7-7718-4028-af30-1825f07dd9d3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:33.916065 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8bbc78-c03e-4518-af96-6a16b77fdbf4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:33.990963 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be465a31-835a-48c7-aff1-362dfb00ecf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:33.991658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be465a31-835a-48c7-aff1-362dfb00ecf8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:33.992006 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be465a31-835a-48c7-aff1-362dfb00ecf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:33.992347 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be465a31-835a-48c7-aff1-362dfb00ecf8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:33.996765 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:34.069897 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d76f7-7718-4028-af30-1825f07dd9d3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:34.122182 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be465a31-835a-48c7-aff1-362dfb00ecf8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:34.125883 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be465a31-835a-48c7-aff1-362dfb00ecf8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:34.140013 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d89dde8-e623-4bff-94c3-4c0608ffcae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:34.141413 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89dde8-e623-4bff-94c3-4c0608ffcae2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:34.141878 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89dde8-e623-4bff-94c3-4c0608ffcae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:34.142345 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89dde8-e623-4bff-94c3-4c0608ffcae2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:34.144271 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d89dde8-e623-4bff-94c3-4c0608ffcae2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:34.146897 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d89dde8-e623-4bff-94c3-4c0608ffcae2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:34.149771 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 92/186] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:07:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:34.150721 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d89dde8-e623-4bff-94c3-4c0608ffcae2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:34.281764 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:34.325916 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 95/187] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:07:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 19:07:34.330036 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2d390-ce02-4841-b02a-791cad256d2c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:34.330545 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2d390-ce02-4841-b02a-791cad256d2c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:34.331057 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2d390-ce02-4841-b02a-791cad256d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:34.336979 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d89dde8-e623-4bff-94c3-4c0608ffcae2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:34.365472 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3bd2d390-ce02-4841-b02a-791cad256d2c None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:34.367696 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3bd2d390-ce02-4841-b02a-791cad256d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:34.408517 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3828430-b2b1-4e99-9d17-170063d4d6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:34.409740 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3828430-b2b1-4e99-9d17-170063d4d6aa None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:34.410250 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3828430-b2b1-4e99-9d17-170063d4d6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:34.410758 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3828430-b2b1-4e99-9d17-170063d4d6aa None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:34.412371 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3828430-b2b1-4e99-9d17-170063d4d6aa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:34.412931 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3828430-b2b1-4e99-9d17-170063d4d6aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:34.415769 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3828430-b2b1-4e99-9d17-170063d4d6aa None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:34.417879 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:34.432799 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 93/188] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:07:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:34.436431 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3828430-b2b1-4e99-9d17-170063d4d6aa None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:34.499978 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 96/189] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:07:34.501909 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd2d390-ce02-4841-b02a-791cad256d2c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:34.646206 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3828430-b2b1-4e99-9d17-170063d4d6aa None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:34.683632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd2d390-ce02-4841-b02a-791cad256d2c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:34.710746 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:34.730269 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5611bdb4-b56a-4e7f-bf7d-75a619943117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:34.731823 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611bdb4-b56a-4e7f-bf7d-75a619943117 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:34.732339 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611bdb4-b56a-4e7f-bf7d-75a619943117 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:34.732849 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611bdb4-b56a-4e7f-bf7d-75a619943117 None admin] PATH_INFO: `/v3/projects/09f6174e875f4cf58a2eb5019ef11189` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:34.735867 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5611bdb4-b56a-4e7f-bf7d-75a619943117 None admin] RBAC: Authorizing `identity:get_project(project_id=09f6174e875f4cf58a2eb5019ef11189)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:34.738738 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5611bdb4-b56a-4e7f-bf7d-75a619943117 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:34.742363 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 94/190] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:07:34 2022] GET /identity/v3/projects/09f6174e875f4cf58a2eb5019ef11189 => generated 298 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:34.743183 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffde993-fed0-45ff-a218-5d1f9d8198bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:34.743407 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5611bdb4-b56a-4e7f-bf7d-75a619943117 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:34.744881 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffde993-fed0-45ff-a218-5d1f9d8198bb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:34.745358 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffde993-fed0-45ff-a218-5d1f9d8198bb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:34.745824 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffde993-fed0-45ff-a218-5d1f9d8198bb None admin] PATH_INFO: `/v3/users/af42430d72cf49eda0d60ad908dc340f` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:34.777734 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffde993-fed0-45ff-a218-5d1f9d8198bb None admin] RBAC: Authorizing `identity:get_user(user_id=af42430d72cf49eda0d60ad908dc340f)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:34.781067 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffde993-fed0-45ff-a218-5d1f9d8198bb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:34.784748 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 97/191] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:07:34 2022] GET /identity/v3/users/af42430d72cf49eda0d60ad908dc340f => generated 283 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:34.786125 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffde993-fed0-45ff-a218-5d1f9d8198bb None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:34.831539 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:34.919376 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5611bdb4-b56a-4e7f-bf7d-75a619943117 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:34.977193 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffde993-fed0-45ff-a218-5d1f9d8198bb None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:34.980671 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2af57-bf80-40d0-a836-21edde0667ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:34.981705 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2af57-bf80-40d0-a836-21edde0667ea None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:34.982033 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2af57-bf80-40d0-a836-21edde0667ea None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:34.982353 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2af57-bf80-40d0-a836-21edde0667ea None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:34.984332 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb2af57-bf80-40d0-a836-21edde0667ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:34.987618 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb2af57-bf80-40d0-a836-21edde0667ea None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:35.027868 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 95/192] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Jul 25 19:07:34 2022] POST /identity/v3/users => generated 297 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:07:35.029326 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2af57-bf80-40d0-a836-21edde0667ea None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:35.037870 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434fb60-2f1e-4555-887c-00e216239040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:35.038951 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434fb60-2f1e-4555-887c-00e216239040 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:35.039273 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434fb60-2f1e-4555-887c-00e216239040 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:35.039783 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434fb60-2f1e-4555-887c-00e216239040 None admin] PATH_INFO: `/v3/projects/c1814fe30f064bc1875e3a1eac8ed490` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:35.041506 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0434fb60-2f1e-4555-887c-00e216239040 None admin] RBAC: Authorizing `identity:get_project(project_id=c1814fe30f064bc1875e3a1eac8ed490)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:35.043119 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0434fb60-2f1e-4555-887c-00e216239040 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:35.045254 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 98/193] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:07:34 2022] GET /identity/v3/projects/c1814fe30f064bc1875e3a1eac8ed490 => generated 312 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:35.049006 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434fb60-2f1e-4555-887c-00e216239040 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:35.176190 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:35.234074 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:35.259317 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434fb60-2f1e-4555-887c-00e216239040 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:35.269487 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2af57-bf80-40d0-a836-21edde0667ea None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:35.321278 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf9994d-34ca-45b3-83e1-7ac27b623ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:35.322589 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9994d-34ca-45b3-83e1-7ac27b623ccd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:35.323559 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9994d-34ca-45b3-83e1-7ac27b623ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:35.323851 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9994d-34ca-45b3-83e1-7ac27b623ccd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:35.337232 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e3348-9ce9-4741-89c8-88f9e20164f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:35.338247 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e3348-9ce9-4741-89c8-88f9e20164f3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:35.338540 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e3348-9ce9-4741-89c8-88f9e20164f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:35.338817 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e3348-9ce9-4741-89c8-88f9e20164f3 None admin] PATH_INFO: `/v3/users/af42430d72cf49eda0d60ad908dc340f` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:35.340467 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3e3348-9ce9-4741-89c8-88f9e20164f3 None admin] RBAC: Authorizing `identity:get_user(user_id=af42430d72cf49eda0d60ad908dc340f)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:35.343117 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3e3348-9ce9-4741-89c8-88f9e20164f3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:35.347629 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 96/194] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:07:35 2022] GET /identity/v3/users/af42430d72cf49eda0d60ad908dc340f => generated 283 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:35.347811 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf9994d-34ca-45b3-83e1-7ac27b623ccd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:07:35.348764 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e3348-9ce9-4741-89c8-88f9e20164f3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:35.365811 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-5bf9994d-34ca-45b3-83e1-7ac27b623ccd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 19:07:35.365811 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:07:35.365811 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:07:35.365811 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:07:35.365811 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:07:35.365811 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:07:35.365811 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:07:35.365811 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:07:35.365811 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:07:35.365811 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:07:35.365811 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:07:35.365811 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:07:35.365811 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:07:35.365811 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:07:35.365811 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:07:35.365811 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:07:35.367774 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:07:35.367774 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:07:35.367774 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:07:35.367774 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:07:35.367774 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:07:35.367774 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:07:35.367774 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:07:35.367774 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:07:35.367774 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:07:35.367774 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:07:35.367774 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:07:35.367774 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:07:35.367774 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:07:35.367774 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:07:35.367774 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:07:35.367774 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:07:35.369343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:07:35.369343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:07:35.369343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:07:35.369343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:07:35.369343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:07:35.369343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:07:35.369343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:07:35.369343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:07:35.369343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:07:35.369343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:07:35.369343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 19:07:35.369343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:07:35.373151 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 99/195] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:07:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:07:35.375228 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf9994d-34ca-45b3-83e1-7ac27b623ccd None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:35.541075 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e3348-9ce9-4741-89c8-88f9e20164f3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:35.583824 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:35.602485 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf9994d-34ca-45b3-83e1-7ac27b623ccd None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:35.604445 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8746208-af13-4a91-bf94-e716247003d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:35.606031 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8746208-af13-4a91-bf94-e716247003d5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:35.607543 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8746208-af13-4a91-bf94-e716247003d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:35.609542 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8746208-af13-4a91-bf94-e716247003d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:35.612492 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8746208-af13-4a91-bf94-e716247003d5 None admin] RBAC: Adding query filter params (role.id=ed7fec18968241efa76a0ee769d96da9, scope.project.id=09f6174e875f4cf58a2eb5019ef11189, user.id=35d700f1abf943789efbf8a08bd9b587) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:35.613325 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8746208-af13-4a91-bf94-e716247003d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:35.615594 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8746208-af13-4a91-bf94-e716247003d5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:35.626776 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 97/196] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:07:35 2022] GET /identity/v3/role_assignments?user.id=35d700f1abf943789efbf8a08bd9b587&scope.project.id=09f6174e875f4cf58a2eb5019ef11189&role.id=ed7fec18968241efa76a0ee769d96da9 => generated 608 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:35.628049 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8746208-af13-4a91-bf94-e716247003d5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:35.749062 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4120e0f4-ce19-4b05-8fe1-679bc7b2631d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:35.750282 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120e0f4-ce19-4b05-8fe1-679bc7b2631d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:35.750813 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120e0f4-ce19-4b05-8fe1-679bc7b2631d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:35.751325 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120e0f4-ce19-4b05-8fe1-679bc7b2631d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:35.836161 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:35.854681 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8746208-af13-4a91-bf94-e716247003d5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:35.918669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8669988-1063-45c7-b94e-673e069386b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:35.920385 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8669988-1063-45c7-b94e-673e069386b7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:35.920385 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8669988-1063-45c7-b94e-673e069386b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:35.920957 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8669988-1063-45c7-b94e-673e069386b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:35.922658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8669988-1063-45c7-b94e-673e069386b7 None admin] RBAC: Adding query filter params (role.id=c241bd9634e24bcbb1854c7621d19f81, scope.project.id=c1814fe30f064bc1875e3a1eac8ed490, user.id=1ee13b9555134b9cafac7f5f0a47204b) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:35.923214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8669988-1063-45c7-b94e-673e069386b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:35.926245 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8669988-1063-45c7-b94e-673e069386b7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:35.939688 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 98/197] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:07:35 2022] GET /identity/v3/role_assignments?user.id=1ee13b9555134b9cafac7f5f0a47204b&scope.project.id=c1814fe30f064bc1875e3a1eac8ed490&role.id=c241bd9634e24bcbb1854c7621d19f81 => generated 608 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:35.940948 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8669988-1063-45c7-b94e-673e069386b7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:36.040445 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4120e0f4-ce19-4b05-8fe1-679bc7b2631d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:36.043318 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4120e0f4-ce19-4b05-8fe1-679bc7b2631d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:36.126583 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:36.250647 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8669988-1063-45c7-b94e-673e069386b7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:36.267780 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 100/198] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:07:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 894 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 19:07:36.272764 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4120e0f4-ce19-4b05-8fe1-679bc7b2631d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:36.310768 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbce471-e9f2-41b6-8cde-3fef2fda1a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:36.312490 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbce471-e9f2-41b6-8cde-3fef2fda1a9b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:36.313121 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbce471-e9f2-41b6-8cde-3fef2fda1a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:36.313773 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbce471-e9f2-41b6-8cde-3fef2fda1a9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:36.340072 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:36.440853 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4120e0f4-ce19-4b05-8fe1-679bc7b2631d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:36.458491 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbce471-e9f2-41b6-8cde-3fef2fda1a9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:36.462013 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbce471-e9f2-41b6-8cde-3fef2fda1a9b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:36.515221 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd180d-01d4-44d0-86f0-4bdfea3deb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:36.516574 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd180d-01d4-44d0-86f0-4bdfea3deb68 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:36.517094 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd180d-01d4-44d0-86f0-4bdfea3deb68 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:36.517609 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd180d-01d4-44d0-86f0-4bdfea3deb68 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:36.519250 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcd180d-01d4-44d0-86f0-4bdfea3deb68 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:36.519691 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcd180d-01d4-44d0-86f0-4bdfea3deb68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:36.522397 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcd180d-01d4-44d0-86f0-4bdfea3deb68 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:36.542492 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 101/199] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Jul 25 19:07:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:36.543966 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd180d-01d4-44d0-86f0-4bdfea3deb68 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:36.600104 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:36.629232 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 99/200] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:07:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 689 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 19:07:36.631213 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbce471-e9f2-41b6-8cde-3fef2fda1a9b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:36.767534 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd180d-01d4-44d0-86f0-4bdfea3deb68 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:36.796774 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:36.821628 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbce471-e9f2-41b6-8cde-3fef2fda1a9b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:36.848570 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dab2302-d99c-468c-ab75-18351a6b4ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:36.850871 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab2302-d99c-468c-ab75-18351a6b4ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:36.851857 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab2302-d99c-468c-ab75-18351a6b4ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:36.852676 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab2302-d99c-468c-ab75-18351a6b4ab3 None admin] PATH_INFO: `/v3/projects/0e408b6ba932474fb12e507cab566abd` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:36.855107 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dab2302-d99c-468c-ab75-18351a6b4ab3 None admin] RBAC: Authorizing `identity:get_project(project_id=0e408b6ba932474fb12e507cab566abd)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:36.858715 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dab2302-d99c-468c-ab75-18351a6b4ab3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:36.862961 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 102/201] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:07:36 2022] GET /identity/v3/projects/0e408b6ba932474fb12e507cab566abd => generated 302 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:36.865395 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dab2302-d99c-468c-ab75-18351a6b4ab3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:36.890071 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d875563-256e-4d0e-991e-50dc74294173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:36.891046 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d875563-256e-4d0e-991e-50dc74294173 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:36.891361 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d875563-256e-4d0e-991e-50dc74294173 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:36.891763 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d875563-256e-4d0e-991e-50dc74294173 None admin] PATH_INFO: `/v3/users/7fc422bf660c4fd1b37fbcc1488b9173` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:36.893461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d875563-256e-4d0e-991e-50dc74294173 None admin] RBAC: Authorizing `identity:get_user(user_id=7fc422bf660c4fd1b37fbcc1488b9173)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:36.895277 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d875563-256e-4d0e-991e-50dc74294173 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:36.897701 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 100/202] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:07:36 2022] GET /identity/v3/users/7fc422bf660c4fd1b37fbcc1488b9173 => generated 289 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:36.898603 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d875563-256e-4d0e-991e-50dc74294173 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:36.929455 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:37.064902 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d875563-256e-4d0e-991e-50dc74294173 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:37.065132 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dab2302-d99c-468c-ab75-18351a6b4ab3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:37.119785 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092da70d-a0f1-46a6-bc81-89e563c36f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:37.120850 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092da70d-a0f1-46a6-bc81-89e563c36f1c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:37.121251 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092da70d-a0f1-46a6-bc81-89e563c36f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:37.121547 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092da70d-a0f1-46a6-bc81-89e563c36f1c None admin] PATH_INFO: `/v3/projects/0e408b6ba932474fb12e507cab566abd` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:37.123024 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092da70d-a0f1-46a6-bc81-89e563c36f1c None admin] RBAC: Authorizing `identity:get_project(project_id=0e408b6ba932474fb12e507cab566abd)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:37.125112 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092da70d-a0f1-46a6-bc81-89e563c36f1c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:37.127078 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 103/203] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:07:36 2022] GET /identity/v3/projects/0e408b6ba932474fb12e507cab566abd => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:37.128038 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092da70d-a0f1-46a6-bc81-89e563c36f1c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:37.130823 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a54688-3e9e-45e6-b83c-a12b236dddfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:37.132037 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a54688-3e9e-45e6-b83c-a12b236dddfa None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:37.132587 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a54688-3e9e-45e6-b83c-a12b236dddfa None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:37.133013 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a54688-3e9e-45e6-b83c-a12b236dddfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:37.134661 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a54688-3e9e-45e6-b83c-a12b236dddfa None admin] RBAC: Adding query filter params (role.id=ed7fec18968241efa76a0ee769d96da9, scope.project.id=0e408b6ba932474fb12e507cab566abd, user.id=af42430d72cf49eda0d60ad908dc340f) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:37.135273 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a54688-3e9e-45e6-b83c-a12b236dddfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:37.138275 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a54688-3e9e-45e6-b83c-a12b236dddfa None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:37.149971 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 101/204] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:07:36 2022] GET /identity/v3/role_assignments?user.id=af42430d72cf49eda0d60ad908dc340f&scope.project.id=0e408b6ba932474fb12e507cab566abd&role.id=ed7fec18968241efa76a0ee769d96da9 => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:37.151666 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a54688-3e9e-45e6-b83c-a12b236dddfa None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:37.217144 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:37.297475 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092da70d-a0f1-46a6-bc81-89e563c36f1c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:37.347943 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e987e72a-6661-4e9b-aa4b-7c3f76e367fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:37.348703 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987e72a-6661-4e9b-aa4b-7c3f76e367fe None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:37.349009 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987e72a-6661-4e9b-aa4b-7c3f76e367fe None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:37.349305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987e72a-6661-4e9b-aa4b-7c3f76e367fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:37.375967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:37.437753 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a54688-3e9e-45e6-b83c-a12b236dddfa None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:37.507150 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e987e72a-6661-4e9b-aa4b-7c3f76e367fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:37.510472 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e987e72a-6661-4e9b-aa4b-7c3f76e367fe None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:37.553708 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e21e7-7b03-48b4-85fa-567d0eacda78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:37.554806 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e21e7-7b03-48b4-85fa-567d0eacda78 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:37.555364 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e21e7-7b03-48b4-85fa-567d0eacda78 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:37.555864 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e21e7-7b03-48b4-85fa-567d0eacda78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:37.557456 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e21e7-7b03-48b4-85fa-567d0eacda78 None admin] RBAC: Adding query filter params (role.id=191e7e31dad148b7a0eb52d3afb5e50e, scope.project.id=0e408b6ba932474fb12e507cab566abd, user.id=af42430d72cf49eda0d60ad908dc340f) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:37.557992 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e21e7-7b03-48b4-85fa-567d0eacda78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:37.560729 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e21e7-7b03-48b4-85fa-567d0eacda78 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:37.580246 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 102/205] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:07:37 2022] GET /identity/v3/role_assignments?user.id=af42430d72cf49eda0d60ad908dc340f&scope.project.id=0e408b6ba932474fb12e507cab566abd&role.id=191e7e31dad148b7a0eb52d3afb5e50e => generated 259 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:37.612013 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:37.711177 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 104/206] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:07:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 19:07:37.718893 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e21e7-7b03-48b4-85fa-567d0eacda78 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:37.950076 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e21e7-7b03-48b4-85fa-567d0eacda78 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:38.018483 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe9e0bf-97b3-4a24-b4af-ff6d77b5bbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:38.020113 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9e0bf-97b3-4a24-b4af-ff6d77b5bbee None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:38.020626 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9e0bf-97b3-4a24-b4af-ff6d77b5bbee None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:38.021127 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9e0bf-97b3-4a24-b4af-ff6d77b5bbee None admin] PATH_INFO: `/v3/projects/09f6174e875f4cf58a2eb5019ef11189` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:38.023538 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe9e0bf-97b3-4a24-b4af-ff6d77b5bbee None admin] RBAC: Authorizing `identity:get_project(project_id=09f6174e875f4cf58a2eb5019ef11189)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:38.026470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe9e0bf-97b3-4a24-b4af-ff6d77b5bbee None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:38.031133 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 103/207] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:07:37 2022] GET /identity/v3/projects/09f6174e875f4cf58a2eb5019ef11189 => generated 298 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:38.038468 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e987e72a-6661-4e9b-aa4b-7c3f76e367fe None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:38.272227 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e987e72a-6661-4e9b-aa4b-7c3f76e367fe None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:38.339975 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba22c82-eef2-4311-a07e-b6b9076858a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:38.341112 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba22c82-eef2-4311-a07e-b6b9076858a6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:38.341871 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba22c82-eef2-4311-a07e-b6b9076858a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:38.342426 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba22c82-eef2-4311-a07e-b6b9076858a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:38.344159 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba22c82-eef2-4311-a07e-b6b9076858a6 None admin] RBAC: Adding query filter params (role.id=6327898c72b044889755dec9746f54dd, scope.project.id=09f6174e875f4cf58a2eb5019ef11189, user.id=7fc422bf660c4fd1b37fbcc1488b9173) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:38.344738 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba22c82-eef2-4311-a07e-b6b9076858a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:38.347587 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba22c82-eef2-4311-a07e-b6b9076858a6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:38.360269 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 105/208] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:07:38 2022] GET /identity/v3/role_assignments?user.id=7fc422bf660c4fd1b37fbcc1488b9173&scope.project.id=09f6174e875f4cf58a2eb5019ef11189&role.id=6327898c72b044889755dec9746f54dd => generated 608 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:38.753105 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced25f00-24c2-429c-a2a5-b0a88df3e65e None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:38.753909 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced25f00-24c2-429c-a2a5-b0a88df3e65e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:38.753909 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced25f00-24c2-429c-a2a5-b0a88df3e65e None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:38.755458 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 104/209] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:07:38.762959 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be97877-5bfe-450e-81ec-e9e4390171be None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:38.763475 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be97877-5bfe-450e-81ec-e9e4390171be None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:38.764034 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be97877-5bfe-450e-81ec-e9e4390171be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:38.811688 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9906421-7fd5-4fc8-96ac-b9086025edea None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:38.812224 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9906421-7fd5-4fc8-96ac-b9086025edea None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:38.812737 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9906421-7fd5-4fc8-96ac-b9086025edea None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:38.815729 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 105/210] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:07:38.820998 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4be97877-5bfe-450e-81ec-e9e4390171be None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:38.822983 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4be97877-5bfe-450e-81ec-e9e4390171be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:38.825229 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1cec2-df11-4410-83c3-220137792542 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:38.826196 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1cec2-df11-4410-83c3-220137792542 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:38.827399 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1cec2-df11-4410-83c3-220137792542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:38.837623 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:38.872072 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-28e1cec2-df11-4410-83c3-220137792542 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:38.875549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-28e1cec2-df11-4410-83c3-220137792542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:38.896561 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:39.000190 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 106/211] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:07:39.049745 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be97877-5bfe-450e-81ec-e9e4390171be None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:39.060608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 106/212] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 19:07:39.104827 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1cec2-df11-4410-83c3-220137792542 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:39.243097 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be97877-5bfe-450e-81ec-e9e4390171be None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:39.326661 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1cec2-df11-4410-83c3-220137792542 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:39.331737 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a055b43-2cfe-4122-a41d-b8640999b957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:39.334239 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a055b43-2cfe-4122-a41d-b8640999b957 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:39.335069 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a055b43-2cfe-4122-a41d-b8640999b957 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:39.335871 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a055b43-2cfe-4122-a41d-b8640999b957 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:39.359720 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a055b43-2cfe-4122-a41d-b8640999b957 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:07:39.369443 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-8a055b43-2cfe-4122-a41d-b8640999b957 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 19:07:39.369443 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:07:39.369443 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:07:39.369443 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:07:39.369443 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:07:39.369443 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:07:39.369443 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:07:39.369443 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:07:39.369443 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:07:39.369443 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:07:39.369443 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:07:39.369443 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:07:39.369443 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:07:39.369443 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:07:39.369443 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:07:39.369443 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:07:39.370991 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:07:39.370991 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:07:39.370991 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:07:39.370991 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:07:39.370991 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:07:39.370991 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:07:39.370991 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:07:39.370991 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:07:39.370991 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:07:39.370991 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:07:39.370991 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:07:39.370991 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:07:39.370991 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:07:39.370991 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:07:39.370991 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:07:39.370991 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:07:39.372147 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:07:39.372147 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:07:39.372147 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:07:39.372147 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:07:39.372147 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:07:39.372147 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:07:39.372147 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:07:39.372147 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:07:39.372147 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:07:39.372147 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:07:39.372147 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 19:07:39.372147 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:07:39.375456 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 107/213] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:07:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:07:39.386117 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a055b43-2cfe-4122-a41d-b8640999b957 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:39.391793 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:39.403210 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d48df6-9232-4862-8d5f-823c72b979ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:39.407240 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d48df6-9232-4862-8d5f-823c72b979ef None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:39.407992 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d48df6-9232-4862-8d5f-823c72b979ef None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:39.408745 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d48df6-9232-4862-8d5f-823c72b979ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:39.411011 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d48df6-9232-4862-8d5f-823c72b979ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:39.413597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d48df6-9232-4862-8d5f-823c72b979ef None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:39.417518 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 107/214] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:07:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:39.426373 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d48df6-9232-4862-8d5f-823c72b979ef None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:39.453729 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:39.602851 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a055b43-2cfe-4122-a41d-b8640999b957 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:39.663197 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d48df6-9232-4862-8d5f-823c72b979ef None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:39.686644 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc6fdbf-2ec2-4b46-ab7c-f429425e7a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:39.688966 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc6fdbf-2ec2-4b46-ab7c-f429425e7a59 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:39.689607 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc6fdbf-2ec2-4b46-ab7c-f429425e7a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:39.690247 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc6fdbf-2ec2-4b46-ab7c-f429425e7a59 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:39.691945 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc6fdbf-2ec2-4b46-ab7c-f429425e7a59 None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:39.692655 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc6fdbf-2ec2-4b46-ab7c-f429425e7a59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:39.695752 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc6fdbf-2ec2-4b46-ab7c-f429425e7a59 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:39.715165 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 108/215] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Jul 25 19:07:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:39.724991 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc6fdbf-2ec2-4b46-ab7c-f429425e7a59 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:39.751722 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcadbaab-14ec-4450-8139-543081c6fbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:39.752988 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcadbaab-14ec-4450-8139-543081c6fbec None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:39.753519 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcadbaab-14ec-4450-8139-543081c6fbec None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:39.754422 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcadbaab-14ec-4450-8139-543081c6fbec None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:39.755636 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcadbaab-14ec-4450-8139-543081c6fbec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:39.757823 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcadbaab-14ec-4450-8139-543081c6fbec None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:39.814551 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 108/216] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Jul 25 19:07:39 2022] POST /identity/v3/users => generated 297 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:07:39.902123 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:39.995616 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc6fdbf-2ec2-4b46-ab7c-f429425e7a59 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:40.048722 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6126374-ca8a-470c-9e7d-7e12d1cc5497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:40.049651 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6126374-ca8a-470c-9e7d-7e12d1cc5497 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:40.049937 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6126374-ca8a-470c-9e7d-7e12d1cc5497 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:40.050216 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6126374-ca8a-470c-9e7d-7e12d1cc5497 None admin] PATH_INFO: `/v3/users/f71d2f3680bd44519cb14fcb9545d89a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:40.071312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6126374-ca8a-470c-9e7d-7e12d1cc5497 None admin] RBAC: Authorizing `identity:get_user(user_id=f71d2f3680bd44519cb14fcb9545d89a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:40.073776 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6126374-ca8a-470c-9e7d-7e12d1cc5497 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:40.076119 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 109/217] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:07:39 2022] GET /identity/v3/users/f71d2f3680bd44519cb14fcb9545d89a => generated 297 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:07:40.085380 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcadbaab-14ec-4450-8139-543081c6fbec None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:40.235546 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:40.313985 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcadbaab-14ec-4450-8139-543081c6fbec None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:40.394187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e403d1db-f7dd-4b38-96bc-00cf26ef82d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:40.395370 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403d1db-f7dd-4b38-96bc-00cf26ef82d8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:40.395957 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403d1db-f7dd-4b38-96bc-00cf26ef82d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:40.396669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403d1db-f7dd-4b38-96bc-00cf26ef82d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:40.556047 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e403d1db-f7dd-4b38-96bc-00cf26ef82d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:40.558571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e403d1db-f7dd-4b38-96bc-00cf26ef82d8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:40.569505 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a83ef6-3a39-42b1-b411-3eedb1b6e888 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:40.571351 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a83ef6-3a39-42b1-b411-3eedb1b6e888 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:40.572190 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a83ef6-3a39-42b1-b411-3eedb1b6e888 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:40.576158 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 110/218] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:40 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:07:40.587568 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9f8a0-70ba-4bd9-8652-f2f169228b4b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:40.588589 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9f8a0-70ba-4bd9-8652-f2f169228b4b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:40.589614 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9f8a0-70ba-4bd9-8652-f2f169228b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:40.657543 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-dfe9f8a0-70ba-4bd9-8652-f2f169228b4b None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:40.661232 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-dfe9f8a0-70ba-4bd9-8652-f2f169228b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:40.824267 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:40.913259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 111/219] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 19:07:40.952611 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 109/220] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:07:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 868 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 19:07:40.962162 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9f8a0-70ba-4bd9-8652-f2f169228b4b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:40.980237 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e403d1db-f7dd-4b38-96bc-00cf26ef82d8 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:41.149422 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:41.320684 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e403d1db-f7dd-4b38-96bc-00cf26ef82d8 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:41.367422 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9f8a0-70ba-4bd9-8652-f2f169228b4b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:41.419807 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec07c612-09f4-4284-8e99-6cd40e4225da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:41.422417 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07c612-09f4-4284-8e99-6cd40e4225da None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:41.424232 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07c612-09f4-4284-8e99-6cd40e4225da None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:41.425041 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07c612-09f4-4284-8e99-6cd40e4225da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:41.453903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec07c612-09f4-4284-8e99-6cd40e4225da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:07:41.469734 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-ec07c612-09f4-4284-8e99-6cd40e4225da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:07:41.469734 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:07:41.469734 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:07:41.469734 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:07:41.469734 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:07:41.469734 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:07:41.469734 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:07:41.469734 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:07:41.469734 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:07:41.469734 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:07:41.469734 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:07:41.469734 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:07:41.469734 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:07:41.469734 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:07:41.469734 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:07:41.469734 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:07:41.471461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:07:41.471461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:07:41.471461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:07:41.471461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:07:41.471461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:07:41.471461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:07:41.471461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:07:41.471461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:07:41.471461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:07:41.471461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:07:41.471461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:07:41.471461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:07:41.471461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:07:41.471461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:07:41.471461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:07:41.471461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:07:41.472333 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:07:41.472333 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:07:41.472333 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:07:41.472333 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:07:41.472333 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:07:41.472333 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:07:41.472333 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:07:41.472333 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:07:41.472333 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:07:41.472333 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:07:41.472333 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:07:41.472333 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:07:41.476303 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0a3b3d-b31d-46ed-90e7-8fd06e5f6d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:41.476613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 110/221] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Jul 25 19:07:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 497 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:07:41.478319 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a3b3d-b31d-46ed-90e7-8fd06e5f6d48 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:41.478977 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a3b3d-b31d-46ed-90e7-8fd06e5f6d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:41.479548 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a3b3d-b31d-46ed-90e7-8fd06e5f6d48 None admin] PATH_INFO: `/v3/projects/0e408b6ba932474fb12e507cab566abd` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:41.479714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c3c60-e735-45bb-a365-8fd45ac6c131 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:41.480375 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c3c60-e735-45bb-a365-8fd45ac6c131 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:41.481256 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c3c60-e735-45bb-a365-8fd45ac6c131 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:41.484699 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 111/222] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:07:41.485206 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0a3b3d-b31d-46ed-90e7-8fd06e5f6d48 None admin] RBAC: Authorizing `identity:get_project(project_id=0e408b6ba932474fb12e507cab566abd)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:41.487016 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0a3b3d-b31d-46ed-90e7-8fd06e5f6d48 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:41.487247 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c3c60-e735-45bb-a365-8fd45ac6c131 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:41.489655 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 112/223] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:07:40 2022] GET /identity/v3/projects/0e408b6ba932474fb12e507cab566abd => generated 302 bytes in 529 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:41.496601 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e170f-666b-4186-8978-fb59173eb593 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:41.496887 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e170f-666b-4186-8978-fb59173eb593 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:41.497415 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e170f-666b-4186-8978-fb59173eb593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:41.529159 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-cd0e170f-666b-4186-8978-fb59173eb593 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:41.532372 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cd0e170f-666b-4186-8978-fb59173eb593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:41.543577 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:41.680442 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 113/224] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 19:07:41.682508 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e170f-666b-4186-8978-fb59173eb593 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:41.724323 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c3c60-e735-45bb-a365-8fd45ac6c131 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:41.787036 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:41.803540 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317aea52-73c0-4c6b-b7f4-633fb3adf8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:41.804832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317aea52-73c0-4c6b-b7f4-633fb3adf8b6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:41.805392 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317aea52-73c0-4c6b-b7f4-633fb3adf8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:41.805953 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317aea52-73c0-4c6b-b7f4-633fb3adf8b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:41.807789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317aea52-73c0-4c6b-b7f4-633fb3adf8b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:41.808414 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317aea52-73c0-4c6b-b7f4-633fb3adf8b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:41.811782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317aea52-73c0-4c6b-b7f4-633fb3adf8b6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:41.835696 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 112/225] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:07:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:41.837859 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317aea52-73c0-4c6b-b7f4-633fb3adf8b6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:41.902557 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e170f-666b-4186-8978-fb59173eb593 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:41.997672 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582e34f-3fec-4d50-8e6b-60bf91ee7a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:41.999496 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582e34f-3fec-4d50-8e6b-60bf91ee7a76 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:41.999871 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582e34f-3fec-4d50-8e6b-60bf91ee7a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:42.000521 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582e34f-3fec-4d50-8e6b-60bf91ee7a76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:42.002121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a582e34f-3fec-4d50-8e6b-60bf91ee7a76 None admin] RBAC: Adding query filter params (role.id=c241bd9634e24bcbb1854c7621d19f81, scope.project.id=0e408b6ba932474fb12e507cab566abd, user.id=f71d2f3680bd44519cb14fcb9545d89a) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:42.002670 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a582e34f-3fec-4d50-8e6b-60bf91ee7a76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:42.006851 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a582e34f-3fec-4d50-8e6b-60bf91ee7a76 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:42.013030 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:42.027267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 114/226] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:07:41 2022] GET /identity/v3/role_assignments?user.id=f71d2f3680bd44519cb14fcb9545d89a&scope.project.id=0e408b6ba932474fb12e507cab566abd&role.id=c241bd9634e24bcbb1854c7621d19f81 => generated 259 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:42.027267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582e34f-3fec-4d50-8e6b-60bf91ee7a76 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:42.058289 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317aea52-73c0-4c6b-b7f4-633fb3adf8b6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:42.181206 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2b186-fbcc-44db-a78b-2abea8eea1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:42.183682 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2b186-fbcc-44db-a78b-2abea8eea1ec None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:42.183682 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2b186-fbcc-44db-a78b-2abea8eea1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:42.183682 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2b186-fbcc-44db-a78b-2abea8eea1ec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:42.223279 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe2b186-fbcc-44db-a78b-2abea8eea1ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:07:42.234870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-bfe2b186-fbcc-44db-a78b-2abea8eea1ec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 19:07:42.234870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:07:42.234870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:07:42.234870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:07:42.234870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:07:42.234870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:07:42.234870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:07:42.234870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:07:42.234870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:07:42.234870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:07:42.234870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:07:42.234870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:07:42.234870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:07:42.234870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:07:42.234870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:07:42.234870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:07:42.236658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:07:42.236658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:07:42.236658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:07:42.236658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:07:42.236658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:07:42.236658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:07:42.236658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:07:42.236658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:07:42.236658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:07:42.236658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:07:42.236658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:07:42.236658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:07:42.236658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:07:42.236658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:07:42.236658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:07:42.236658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:07:42.238189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:07:42.238189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:07:42.238189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:07:42.238189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:07:42.238189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:07:42.238189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:07:42.238189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:07:42.238189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:07:42.238189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:07:42.238189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:07:42.238189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 19:07:42.238189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:07:42.241641 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 113/227] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Jul 25 19:07:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:07:42.252113 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2b186-fbcc-44db-a78b-2abea8eea1ec None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:42.292376 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582e34f-3fec-4d50-8e6b-60bf91ee7a76 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:42.316242 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:42.346373 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7f6d3-16f6-4b92-bf5e-657e9eb48e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:42.347404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7f6d3-16f6-4b92-bf5e-657e9eb48e57 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:42.347789 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7f6d3-16f6-4b92-bf5e-657e9eb48e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:42.348177 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7f6d3-16f6-4b92-bf5e-657e9eb48e57 None admin] PATH_INFO: `/v3/users/af42430d72cf49eda0d60ad908dc340f` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:42.349742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb7f6d3-16f6-4b92-bf5e-657e9eb48e57 None admin] RBAC: Authorizing `identity:get_user(user_id=af42430d72cf49eda0d60ad908dc340f)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:42.351336 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb7f6d3-16f6-4b92-bf5e-657e9eb48e57 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:42.355301 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 115/228] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:07:42 2022] GET /identity/v3/users/af42430d72cf49eda0d60ad908dc340f => generated 283 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:42.365855 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7f6d3-16f6-4b92-bf5e-657e9eb48e57 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:42.535450 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2b186-fbcc-44db-a78b-2abea8eea1ec None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:42.660627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3f539-06e8-4b0f-bd3f-4845c5ec567f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:42.662467 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3f539-06e8-4b0f-bd3f-4845c5ec567f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:42.662940 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3f539-06e8-4b0f-bd3f-4845c5ec567f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:42.663394 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3f539-06e8-4b0f-bd3f-4845c5ec567f None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:42.665540 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e3f539-06e8-4b0f-bd3f-4845c5ec567f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:42.666122 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e3f539-06e8-4b0f-bd3f-4845c5ec567f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:42.668777 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e3f539-06e8-4b0f-bd3f-4845c5ec567f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:42.690513 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 114/229] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:07:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 439 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:42.697168 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3f539-06e8-4b0f-bd3f-4845c5ec567f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:42.720516 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7f6d3-16f6-4b92-bf5e-657e9eb48e57 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:42.729360 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:42.869141 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1878750-a3e0-42a8-a85c-3a0deacc4542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:42.871396 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1878750-a3e0-42a8-a85c-3a0deacc4542 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:42.872732 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1878750-a3e0-42a8-a85c-3a0deacc4542 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:42.874291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1878750-a3e0-42a8-a85c-3a0deacc4542 None admin] PATH_INFO: `/v3/projects/0e408b6ba932474fb12e507cab566abd` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:42.882607 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1878750-a3e0-42a8-a85c-3a0deacc4542 None admin] RBAC: Authorizing `identity:get_project(project_id=0e408b6ba932474fb12e507cab566abd)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:42.886151 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1878750-a3e0-42a8-a85c-3a0deacc4542 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:42.893775 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 116/230] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:07:42 2022] GET /identity/v3/projects/0e408b6ba932474fb12e507cab566abd => generated 302 bytes in 529 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:42.903701 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1878750-a3e0-42a8-a85c-3a0deacc4542 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:42.909817 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3f539-06e8-4b0f-bd3f-4845c5ec567f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:43.006028 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f32e23a-05a3-4f45-8df8-4041bace4999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:43.007844 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32e23a-05a3-4f45-8df8-4041bace4999 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:43.008150 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32e23a-05a3-4f45-8df8-4041bace4999 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:43.008437 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32e23a-05a3-4f45-8df8-4041bace4999 None admin] PATH_INFO: `/v3/users/af42430d72cf49eda0d60ad908dc340f` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:43.012682 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f32e23a-05a3-4f45-8df8-4041bace4999 None admin] RBAC: Authorizing `identity:get_user(user_id=af42430d72cf49eda0d60ad908dc340f)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:43.014981 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f32e23a-05a3-4f45-8df8-4041bace4999 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:43.019210 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 115/231] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:07:42 2022] GET /identity/v3/users/af42430d72cf49eda0d60ad908dc340f => generated 283 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:43.025925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f32e23a-05a3-4f45-8df8-4041bace4999 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:43.044247 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:43.119705 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1878750-a3e0-42a8-a85c-3a0deacc4542 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:43.184699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7810fd4a-5fde-4d4a-ac2c-fc30e2bdbfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:43.186307 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810fd4a-5fde-4d4a-ac2c-fc30e2bdbfdb None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:43.186794 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810fd4a-5fde-4d4a-ac2c-fc30e2bdbfdb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:43.187286 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810fd4a-5fde-4d4a-ac2c-fc30e2bdbfdb None admin] PATH_INFO: `/v3/projects/0e408b6ba932474fb12e507cab566abd/users/af42430d72cf49eda0d60ad908dc340f/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:43.201186 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7810fd4a-5fde-4d4a-ac2c-fc30e2bdbfdb None admin] RBAC: Authorizing `identity:create_grant(project_id=0e408b6ba932474fb12e507cab566abd, user_id=af42430d72cf49eda0d60ad908dc340f, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:43.204232 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7810fd4a-5fde-4d4a-ac2c-fc30e2bdbfdb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:43.219741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:43.246363 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 117/232] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Jul 25 19:07:42 2022] PUT /identity/v3/projects/0e408b6ba932474fb12e507cab566abd/users/af42430d72cf49eda0d60ad908dc340f/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:07:43.283938 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f32e23a-05a3-4f45-8df8-4041bace4999 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:43.358865 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8f185-95e8-474c-b85c-b329ce3b2d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:43.360598 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8f185-95e8-474c-b85c-b329ce3b2d7b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:43.361167 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8f185-95e8-474c-b85c-b329ce3b2d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:43.361670 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8f185-95e8-474c-b85c-b329ce3b2d7b None admin] PATH_INFO: `/v3/projects/0e408b6ba932474fb12e507cab566abd` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:43.364045 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad8f185-95e8-474c-b85c-b329ce3b2d7b None admin] RBAC: Authorizing `identity:get_project(project_id=0e408b6ba932474fb12e507cab566abd)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:43.367313 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad8f185-95e8-474c-b85c-b329ce3b2d7b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:43.371241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 116/233] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:07:43 2022] GET /identity/v3/projects/0e408b6ba932474fb12e507cab566abd => generated 302 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:07:43.380628 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7810fd4a-5fde-4d4a-ac2c-fc30e2bdbfdb None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:43.595484 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7810fd4a-5fde-4d4a-ac2c-fc30e2bdbfdb None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:43.618418 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:43.671024 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca86e83-9cdf-4639-a9b7-de8d953af80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:43.671901 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca86e83-9cdf-4639-a9b7-de8d953af80e None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:43.672310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca86e83-9cdf-4639-a9b7-de8d953af80e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:43.672596 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca86e83-9cdf-4639-a9b7-de8d953af80e None admin] PATH_INFO: `/v3/projects/0e408b6ba932474fb12e507cab566abd/users/af42430d72cf49eda0d60ad908dc340f/roles/191e7e31dad148b7a0eb52d3afb5e50e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:43.677060 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca86e83-9cdf-4639-a9b7-de8d953af80e None admin] RBAC: Authorizing `identity:create_grant(project_id=0e408b6ba932474fb12e507cab566abd, user_id=af42430d72cf49eda0d60ad908dc340f, role_id=191e7e31dad148b7a0eb52d3afb5e50e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:43.678579 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca86e83-9cdf-4639-a9b7-de8d953af80e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:43.702558 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 118/234] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Jul 25 19:07:43 2022] PUT /identity/v3/projects/0e408b6ba932474fb12e507cab566abd/users/af42430d72cf49eda0d60ad908dc340f/roles/191e7e31dad148b7a0eb52d3afb5e50e => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:07:43.870646 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0940c-0db8-4050-b2ba-3dbc682481c2 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:43.871136 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0940c-0db8-4050-b2ba-3dbc682481c2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:43.871674 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0940c-0db8-4050-b2ba-3dbc682481c2 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:43.874959 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 117/235] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:07:43.884681 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e6674-b3ac-471e-8dd4-d5f0afe8362c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:43.885478 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e6674-b3ac-471e-8dd4-d5f0afe8362c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:43.885829 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e6674-b3ac-471e-8dd4-d5f0afe8362c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:43.922970 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-960e6674-b3ac-471e-8dd4-d5f0afe8362c None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:43.924474 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-960e6674-b3ac-471e-8dd4-d5f0afe8362c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:43.945642 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55913283-bdae-49cb-bd43-bc2abc706e6b None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:43.946419 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55913283-bdae-49cb-bd43-bc2abc706e6b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:43.946419 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55913283-bdae-49cb-bd43-bc2abc706e6b None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:43.948067 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 118/236] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:07:43.961954 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784f1a9-258e-44ff-ae43-58d540faca55 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:43.962387 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784f1a9-258e-44ff-ae43-58d540faca55 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:43.962628 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784f1a9-258e-44ff-ae43-58d540faca55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:43.999139 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1784f1a9-258e-44ff-ae43-58d540faca55 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:44.000549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1784f1a9-258e-44ff-ae43-58d540faca55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:44.111286 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 119/237] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:07:44.147954 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e6674-b3ac-471e-8dd4-d5f0afe8362c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:44.178577 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 119/238] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 19:07:44.257999 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1784f1a9-258e-44ff-ae43-58d540faca55 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:44.345563 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:44.349719 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e6674-b3ac-471e-8dd4-d5f0afe8362c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:44.395561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba8017-ba84-429c-ba48-49ef74f315db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:44.395561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba8017-ba84-429c-ba48-49ef74f315db None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:44.395561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba8017-ba84-429c-ba48-49ef74f315db None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:44.395561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba8017-ba84-429c-ba48-49ef74f315db None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:44.415577 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ba8017-ba84-429c-ba48-49ef74f315db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:07:44.424390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-39ba8017-ba84-429c-ba48-49ef74f315db None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 19:07:44.424390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:07:44.424390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:07:44.424390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:07:44.424390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:07:44.424390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:07:44.424390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:07:44.424390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:07:44.424390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:07:44.424390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:07:44.424390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:07:44.424390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:07:44.424390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:07:44.424390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:07:44.424390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:07:44.424390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:07:44.426398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:07:44.426398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:07:44.426398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:07:44.426398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:07:44.426398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:07:44.426398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:07:44.426398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:07:44.426398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:07:44.426398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:07:44.426398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:07:44.426398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:07:44.426398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:07:44.426398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:07:44.426398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:07:44.426398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:07:44.426398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:07:44.428023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:07:44.428023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:07:44.428023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:07:44.428023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:07:44.428023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:07:44.428023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:07:44.428023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:07:44.428023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:07:44.428023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:07:44.428023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:07:44.428023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 19:07:44.428023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:07:44.434528 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 120/239] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:07:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:07:44.449851 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba8017-ba84-429c-ba48-49ef74f315db None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:44.565249 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:44.616754 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1784f1a9-258e-44ff-ae43-58d540faca55 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:44.663472 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba8017-ba84-429c-ba48-49ef74f315db None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:44.724932 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49373f26-788d-4ebe-be42-e1c9c5073bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:44.726223 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49373f26-788d-4ebe-be42-e1c9c5073bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:44.726736 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49373f26-788d-4ebe-be42-e1c9c5073bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:44.727241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49373f26-788d-4ebe-be42-e1c9c5073bf5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:44.729725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49373f26-788d-4ebe-be42-e1c9c5073bf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:44.732901 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49373f26-788d-4ebe-be42-e1c9c5073bf5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:44.734742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6180e-3730-4c45-badc-be5461181f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:44.735269 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 120/240] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:07:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 478 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:44.735938 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6180e-3730-4c45-badc-be5461181f27 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:44.736456 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6180e-3730-4c45-badc-be5461181f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:44.737340 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6180e-3730-4c45-badc-be5461181f27 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:44.738796 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a6180e-3730-4c45-badc-be5461181f27 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:44.739339 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a6180e-3730-4c45-badc-be5461181f27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:44.742068 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a6180e-3730-4c45-badc-be5461181f27 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:44.743625 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49373f26-788d-4ebe-be42-e1c9c5073bf5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:44.758635 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 121/241] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Jul 25 19:07:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:44.769568 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6180e-3730-4c45-badc-be5461181f27 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:44.840745 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:44.902628 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49373f26-788d-4ebe-be42-e1c9c5073bf5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:44.950080 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8dd686-2c22-4183-b1c2-980338da1f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:44.952327 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8dd686-2c22-4183-b1c2-980338da1f3d None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:44.953120 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8dd686-2c22-4183-b1c2-980338da1f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:44.953820 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8dd686-2c22-4183-b1c2-980338da1f3d None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:44.955895 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8dd686-2c22-4183-b1c2-980338da1f3d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:44.957692 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8dd686-2c22-4183-b1c2-980338da1f3d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:44.984275 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:44.989360 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 121/242] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Jul 25 19:07:44 2022] POST /identity/v3/users => generated 293 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:07:45.133507 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6180e-3730-4c45-badc-be5461181f27 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:45.221868 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b164516-d3b3-4c67-ab9d-a5804606dedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:45.222813 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b164516-d3b3-4c67-ab9d-a5804606dedf None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:45.223101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b164516-d3b3-4c67-ab9d-a5804606dedf None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:45.223404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b164516-d3b3-4c67-ab9d-a5804606dedf None admin] PATH_INFO: `/v3/users/59d30a538a004005802bb5854af33511` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:45.247786 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b164516-d3b3-4c67-ab9d-a5804606dedf None admin] RBAC: Authorizing `identity:get_user(user_id=59d30a538a004005802bb5854af33511)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:45.250253 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b164516-d3b3-4c67-ab9d-a5804606dedf None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:45.252625 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 122/243] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:07:44 2022] GET /identity/v3/users/59d30a538a004005802bb5854af33511 => generated 297 bytes in 484 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:45.262058 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8dd686-2c22-4183-b1c2-980338da1f3d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:45.410632 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8dd686-2c22-4183-b1c2-980338da1f3d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:45.457695 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e546d177-a16e-4c61-94d1-665c76c1a7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:45.458774 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e546d177-a16e-4c61-94d1-665c76c1a7ad None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:45.459303 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e546d177-a16e-4c61-94d1-665c76c1a7ad None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:45.459856 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e546d177-a16e-4c61-94d1-665c76c1a7ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:45.534292 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3695a8-feb0-4141-a272-265ded50d7c9 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:45.534714 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3695a8-feb0-4141-a272-265ded50d7c9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:45.535213 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3695a8-feb0-4141-a272-265ded50d7c9 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:45.542877 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 123/244] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:45 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:07:45.549587 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66e76f-44c5-428f-aa24-c0f927edb30b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:45.550073 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66e76f-44c5-428f-aa24-c0f927edb30b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:45.550569 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66e76f-44c5-428f-aa24-c0f927edb30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:45.554445 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:45.559654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e546d177-a16e-4c61-94d1-665c76c1a7ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:45.562128 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e546d177-a16e-4c61-94d1-665c76c1a7ad None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:45.592028 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9a66e76f-44c5-428f-aa24-c0f927edb30b None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:45.594446 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9a66e76f-44c5-428f-aa24-c0f927edb30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:45.730555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 122/245] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:07:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 19:07:45.738542 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e546d177-a16e-4c61-94d1-665c76c1a7ad None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:45.744005 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 124/246] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:07:45.801107 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a66e76f-44c5-428f-aa24-c0f927edb30b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:45.823590 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:45.877646 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:45.882799 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e546d177-a16e-4c61-94d1-665c76c1a7ad None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:45.933683 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd02898-c270-43b1-a850-330ebc48e361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:45.934789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd02898-c270-43b1-a850-330ebc48e361 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:45.935077 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd02898-c270-43b1-a850-330ebc48e361 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:45.935347 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd02898-c270-43b1-a850-330ebc48e361 None admin] PATH_INFO: `/v3/projects/0e408b6ba932474fb12e507cab566abd` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:45.938536 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd02898-c270-43b1-a850-330ebc48e361 None admin] RBAC: Authorizing `identity:get_project(project_id=0e408b6ba932474fb12e507cab566abd)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:45.940193 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd02898-c270-43b1-a850-330ebc48e361 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:45.942163 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 123/247] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:07:45 2022] GET /identity/v3/projects/0e408b6ba932474fb12e507cab566abd => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:07:45.948669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd02898-c270-43b1-a850-330ebc48e361 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:45.985601 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a66e76f-44c5-428f-aa24-c0f927edb30b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:46.062137 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e05f2c-9af5-41fd-9251-03fff507f704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:46.063366 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e05f2c-9af5-41fd-9251-03fff507f704 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:46.063689 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e05f2c-9af5-41fd-9251-03fff507f704 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:46.064073 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e05f2c-9af5-41fd-9251-03fff507f704 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:46.080766 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e05f2c-9af5-41fd-9251-03fff507f704 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:07:46.088992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-46e05f2c-9af5-41fd-9251-03fff507f704 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 19:07:46.088992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:07:46.088992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:07:46.088992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:07:46.088992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:07:46.088992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:07:46.088992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:07:46.088992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:07:46.088992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:07:46.088992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:07:46.088992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:07:46.088992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:07:46.088992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:07:46.088992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:07:46.088992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:07:46.088992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:07:46.090154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:07:46.090154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:07:46.090154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:07:46.090154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:07:46.090154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:07:46.090154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:07:46.090154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:07:46.090154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:07:46.090154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:07:46.090154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:07:46.090154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:07:46.090154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:07:46.090154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:07:46.090154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:07:46.090154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:07:46.090154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:07:46.090953 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:07:46.090953 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:07:46.090953 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:07:46.090953 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:07:46.090953 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:07:46.090953 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:07:46.090953 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:07:46.090953 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:07:46.090953 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:07:46.090953 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:07:46.090953 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 19:07:46.090953 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:07:46.090953 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 125/248] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:07:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:07:46.098984 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e05f2c-9af5-41fd-9251-03fff507f704 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:46.119923 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd02898-c270-43b1-a850-330ebc48e361 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:46.231588 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af398c4-5eeb-433f-a553-2f7bb8ff2e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:46.236204 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af398c4-5eeb-433f-a553-2f7bb8ff2e03 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:46.236856 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af398c4-5eeb-433f-a553-2f7bb8ff2e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:46.237328 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af398c4-5eeb-433f-a553-2f7bb8ff2e03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:46.239203 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af398c4-5eeb-433f-a553-2f7bb8ff2e03 None admin] RBAC: Adding query filter params (role.id=6327898c72b044889755dec9746f54dd, scope.project.id=0e408b6ba932474fb12e507cab566abd, user.id=59d30a538a004005802bb5854af33511) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:46.239801 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af398c4-5eeb-433f-a553-2f7bb8ff2e03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:46.248454 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af398c4-5eeb-433f-a553-2f7bb8ff2e03 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:46.284773 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 124/249] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:07:45 2022] GET /identity/v3/role_assignments?user.id=59d30a538a004005802bb5854af33511&scope.project.id=0e408b6ba932474fb12e507cab566abd&role.id=6327898c72b044889755dec9746f54dd => generated 259 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:46.312621 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:46.382008 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e05f2c-9af5-41fd-9251-03fff507f704 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:46.456863 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be938e0b-c291-4afb-bdb7-ab741509f4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:46.458102 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be938e0b-c291-4afb-bdb7-ab741509f4b0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:46.458638 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be938e0b-c291-4afb-bdb7-ab741509f4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:46.459153 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be938e0b-c291-4afb-bdb7-ab741509f4b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:46.460673 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be938e0b-c291-4afb-bdb7-ab741509f4b0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:46.461229 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be938e0b-c291-4afb-bdb7-ab741509f4b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:46.464053 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be938e0b-c291-4afb-bdb7-ab741509f4b0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:46.487356 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 126/250] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Jul 25 19:07:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:46.494322 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65101621-87f4-45e4-be50-6df48f6cdbf8 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:46.494849 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65101621-87f4-45e4-be50-6df48f6cdbf8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:46.495357 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65101621-87f4-45e4-be50-6df48f6cdbf8 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:46.498197 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 125/251] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:07:46.505880 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94262347-1a74-4f2d-a0c2-7bede9cdd37c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:46.506397 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94262347-1a74-4f2d-a0c2-7bede9cdd37c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:46.506921 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94262347-1a74-4f2d-a0c2-7bede9cdd37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:46.512412 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be938e0b-c291-4afb-bdb7-ab741509f4b0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:46.576734 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-94262347-1a74-4f2d-a0c2-7bede9cdd37c None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:46.578436 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-94262347-1a74-4f2d-a0c2-7bede9cdd37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:46.646620 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:46.692322 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be938e0b-c291-4afb-bdb7-ab741509f4b0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:46.731217 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:46.743497 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f69d0-b474-40eb-ac5b-45877c619b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:46.744452 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f69d0-b474-40eb-ac5b-45877c619b62 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:46.744737 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f69d0-b474-40eb-ac5b-45877c619b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:46.745018 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f69d0-b474-40eb-ac5b-45877c619b62 None admin] PATH_INFO: `/v3/users/f71d2f3680bd44519cb14fcb9545d89a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:46.746611 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f69d0-b474-40eb-ac5b-45877c619b62 None admin] RBAC: Authorizing `identity:get_user(user_id=f71d2f3680bd44519cb14fcb9545d89a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:46.748853 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f69d0-b474-40eb-ac5b-45877c619b62 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:46.752687 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 127/252] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:07:46 2022] GET /identity/v3/users/f71d2f3680bd44519cb14fcb9545d89a => generated 297 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:46.760343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f69d0-b474-40eb-ac5b-45877c619b62 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:46.768253 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 126/253] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 19:07:46.867041 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94262347-1a74-4f2d-a0c2-7bede9cdd37c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:46.948173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f69d0-b474-40eb-ac5b-45877c619b62 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:47.045991 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e710dea-0e44-4dec-9fa8-d1a856129ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:47.047028 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e710dea-0e44-4dec-9fa8-d1a856129ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:47.047317 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e710dea-0e44-4dec-9fa8-d1a856129ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:47.047622 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e710dea-0e44-4dec-9fa8-d1a856129ee5 None admin] PATH_INFO: `/v3/projects/0e408b6ba932474fb12e507cab566abd` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:47.049888 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e710dea-0e44-4dec-9fa8-d1a856129ee5 None admin] RBAC: Authorizing `identity:get_project(project_id=0e408b6ba932474fb12e507cab566abd)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:47.051409 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e710dea-0e44-4dec-9fa8-d1a856129ee5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:47.053524 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 128/254] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:07:46 2022] GET /identity/v3/projects/0e408b6ba932474fb12e507cab566abd => generated 302 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:47.062764 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e710dea-0e44-4dec-9fa8-d1a856129ee5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:47.171893 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94262347-1a74-4f2d-a0c2-7bede9cdd37c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:47.220157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:47.253911 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6d82d-dae5-4572-ac7d-af3ac7b84f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:47.254738 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6d82d-dae5-4572-ac7d-af3ac7b84f85 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:47.255021 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6d82d-dae5-4572-ac7d-af3ac7b84f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:47.255298 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6d82d-dae5-4572-ac7d-af3ac7b84f85 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:47.274227 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e710dea-0e44-4dec-9fa8-d1a856129ee5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:47.282505 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b6d82d-dae5-4572-ac7d-af3ac7b84f85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:07:47.297461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-e3b6d82d-dae5-4572-ac7d-af3ac7b84f85 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:07:47.297461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:07:47.297461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:07:47.297461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:07:47.297461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:07:47.297461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:07:47.297461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:07:47.297461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:07:47.297461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:07:47.297461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:07:47.297461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:07:47.297461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:07:47.297461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:07:47.297461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:07:47.297461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:07:47.297461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:07:47.299425 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:07:47.299425 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:07:47.299425 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:07:47.299425 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:07:47.299425 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:07:47.299425 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:07:47.299425 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:07:47.299425 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:07:47.299425 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:07:47.299425 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:07:47.299425 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:07:47.299425 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:07:47.299425 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:07:47.299425 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:07:47.299425 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:07:47.299425 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:07:47.301078 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:07:47.301078 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:07:47.301078 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:07:47.301078 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:07:47.301078 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:07:47.301078 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:07:47.301078 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:07:47.301078 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:07:47.301078 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:07:47.301078 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:07:47.301078 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:07:47.301078 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:07:47.302279 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 127/255] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Jul 25 19:07:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 436 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:07:47.314677 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6d82d-dae5-4572-ac7d-af3ac7b84f85 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:47.345788 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236b7160-45cd-427c-ae22-2b219358972b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:47.346704 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236b7160-45cd-427c-ae22-2b219358972b None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:47.347184 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236b7160-45cd-427c-ae22-2b219358972b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:47.347851 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236b7160-45cd-427c-ae22-2b219358972b None admin] PATH_INFO: `/v3/projects/0e408b6ba932474fb12e507cab566abd/users/f71d2f3680bd44519cb14fcb9545d89a/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:47.352615 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236b7160-45cd-427c-ae22-2b219358972b None admin] RBAC: Authorizing `identity:create_grant(project_id=0e408b6ba932474fb12e507cab566abd, user_id=f71d2f3680bd44519cb14fcb9545d89a, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:47.354947 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236b7160-45cd-427c-ae22-2b219358972b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:47.382166 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:47.385696 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 129/256] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Jul 25 19:07:47 2022] PUT /identity/v3/projects/0e408b6ba932474fb12e507cab566abd/users/f71d2f3680bd44519cb14fcb9545d89a/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:07:47.493477 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a24c1a-a6fc-4f6d-9332-8a81aae4eefd None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:47.494214 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a24c1a-a6fc-4f6d-9332-8a81aae4eefd None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:47.494729 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a24c1a-a6fc-4f6d-9332-8a81aae4eefd None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:47.499361 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 130/257] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:07:47.518757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c1a8e-2766-4514-b6e8-0fa4bc7124d5 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:47.519560 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c1a8e-2766-4514-b6e8-0fa4bc7124d5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:47.520089 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c1a8e-2766-4514-b6e8-0fa4bc7124d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:47.570294 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6c6c1a8e-2766-4514-b6e8-0fa4bc7124d5 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:47.581201 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6c6c1a8e-2766-4514-b6e8-0fa4bc7124d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:47.587780 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6d82d-dae5-4572-ac7d-af3ac7b84f85 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:47.715328 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 131/258] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:07:47.721512 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae61acf-f95a-49b1-8dc4-6ed006abade3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:47.722729 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae61acf-f95a-49b1-8dc4-6ed006abade3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:47.723397 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae61acf-f95a-49b1-8dc4-6ed006abade3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:47.723968 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae61acf-f95a-49b1-8dc4-6ed006abade3 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:47.725046 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae61acf-f95a-49b1-8dc4-6ed006abade3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:47.725565 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae61acf-f95a-49b1-8dc4-6ed006abade3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:47.727409 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae61acf-f95a-49b1-8dc4-6ed006abade3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:47.743715 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 128/259] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:07:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:47.749897 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6c1a8e-2766-4514-b6e8-0fa4bc7124d5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:47.781821 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae61acf-f95a-49b1-8dc4-6ed006abade3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:47.883340 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:47.965078 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6c1a8e-2766-4514-b6e8-0fa4bc7124d5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:48.016920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e887b6-a2b5-4545-9dca-a6a4f3d8342e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:48.017768 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e887b6-a2b5-4545-9dca-a6a4f3d8342e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:48.018061 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e887b6-a2b5-4545-9dca-a6a4f3d8342e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:48.018773 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e887b6-a2b5-4545-9dca-a6a4f3d8342e None admin] PATH_INFO: `/v3/users/af42430d72cf49eda0d60ad908dc340f` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:48.021085 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e887b6-a2b5-4545-9dca-a6a4f3d8342e None admin] RBAC: Authorizing `identity:get_user(user_id=af42430d72cf49eda0d60ad908dc340f)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:48.022677 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e887b6-a2b5-4545-9dca-a6a4f3d8342e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:48.024852 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 132/260] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:07:47 2022] GET /identity/v3/users/af42430d72cf49eda0d60ad908dc340f => generated 283 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:48.034157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e887b6-a2b5-4545-9dca-a6a4f3d8342e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:48.054505 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:48.134942 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae61acf-f95a-49b1-8dc4-6ed006abade3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:48.205314 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e887b6-a2b5-4545-9dca-a6a4f3d8342e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:48.247712 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4bcd25-b76e-4f1e-83f7-78e2f74359b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:48.249247 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4bcd25-b76e-4f1e-83f7-78e2f74359b0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:48.249811 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4bcd25-b76e-4f1e-83f7-78e2f74359b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:48.250246 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4bcd25-b76e-4f1e-83f7-78e2f74359b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:48.255855 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:48.288882 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cecbdf0-2745-4884-a8b7-0898f8775edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:48.290149 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cecbdf0-2745-4884-a8b7-0898f8775edd None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:48.290635 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cecbdf0-2745-4884-a8b7-0898f8775edd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:48.291128 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cecbdf0-2745-4884-a8b7-0898f8775edd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:48.329973 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cecbdf0-2745-4884-a8b7-0898f8775edd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:07:48.360543 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4bcd25-b76e-4f1e-83f7-78e2f74359b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:48.363955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4bcd25-b76e-4f1e-83f7-78e2f74359b0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:48.364841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-4cecbdf0-2745-4884-a8b7-0898f8775edd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 19:07:48.364841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:07:48.364841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:07:48.364841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:07:48.364841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:07:48.364841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:07:48.364841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:07:48.364841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:07:48.364841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:07:48.364841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:07:48.364841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:07:48.364841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:07:48.364841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:07:48.364841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:07:48.364841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:07:48.364841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:07:48.366665 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:07:48.366665 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:07:48.366665 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:07:48.366665 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:07:48.366665 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:07:48.366665 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:07:48.366665 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:07:48.366665 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:07:48.366665 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:07:48.366665 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:07:48.366665 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:07:48.366665 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:07:48.366665 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:07:48.366665 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:07:48.366665 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:07:48.366665 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:07:48.368339 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:07:48.368339 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:07:48.368339 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:07:48.368339 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:07:48.368339 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:07:48.368339 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:07:48.368339 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:07:48.368339 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:07:48.368339 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:07:48.368339 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:07:48.368339 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 19:07:48.368339 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:07:48.368339 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 129/261] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Jul 25 19:07:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 586 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:07:48.381360 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cecbdf0-2745-4884-a8b7-0898f8775edd None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:48.711293 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cecbdf0-2745-4884-a8b7-0898f8775edd None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:48.714019 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 133/262] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:07:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 680 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 19:07:48.725912 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4bcd25-b76e-4f1e-83f7-78e2f74359b0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:48.789224 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:48.846440 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfef38d-bc4c-456d-a9f4-93f61b638b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:48.847848 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfef38d-bc4c-456d-a9f4-93f61b638b85 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:48.848471 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfef38d-bc4c-456d-a9f4-93f61b638b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:48.849097 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfef38d-bc4c-456d-a9f4-93f61b638b85 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:48.850937 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfef38d-bc4c-456d-a9f4-93f61b638b85 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:48.851867 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfef38d-bc4c-456d-a9f4-93f61b638b85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:48.863550 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfef38d-bc4c-456d-a9f4-93f61b638b85 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:48.885797 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 130/263] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:07:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 505 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:48.895482 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfef38d-bc4c-456d-a9f4-93f61b638b85 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:48.929586 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:48.952814 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4bcd25-b76e-4f1e-83f7-78e2f74359b0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:49.025586 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76176933-4c9a-49e5-a5bc-a4f185a68a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:49.027350 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76176933-4c9a-49e5-a5bc-a4f185a68a95 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:49.028166 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76176933-4c9a-49e5-a5bc-a4f185a68a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:49.028692 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76176933-4c9a-49e5-a5bc-a4f185a68a95 None admin] PATH_INFO: `/v3/projects/0e408b6ba932474fb12e507cab566abd` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:49.040052 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76176933-4c9a-49e5-a5bc-a4f185a68a95 None admin] RBAC: Authorizing `identity:get_project(project_id=0e408b6ba932474fb12e507cab566abd)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:49.042101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76176933-4c9a-49e5-a5bc-a4f185a68a95 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:49.044394 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 134/264] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:07:48 2022] GET /identity/v3/projects/0e408b6ba932474fb12e507cab566abd => generated 302 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:07:49.056331 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76176933-4c9a-49e5-a5bc-a4f185a68a95 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:49.213923 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfef38d-bc4c-456d-a9f4-93f61b638b85 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:49.292572 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c0f2c-b76f-49fd-9e1c-abf12033c740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:49.296401 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c0f2c-b76f-49fd-9e1c-abf12033c740 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:49.297494 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c0f2c-b76f-49fd-9e1c-abf12033c740 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:49.298225 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c0f2c-b76f-49fd-9e1c-abf12033c740 None admin] PATH_INFO: `/v3/users/af42430d72cf49eda0d60ad908dc340f` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:49.301308 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4c0f2c-b76f-49fd-9e1c-abf12033c740 None admin] RBAC: Authorizing `identity:get_user(user_id=af42430d72cf49eda0d60ad908dc340f)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:49.304227 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4c0f2c-b76f-49fd-9e1c-abf12033c740 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:49.309723 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 131/265] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:07:48 2022] GET /identity/v3/users/af42430d72cf49eda0d60ad908dc340f => generated 283 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:49.310328 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:49.318033 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c0f2c-b76f-49fd-9e1c-abf12033c740 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:49.325394 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76176933-4c9a-49e5-a5bc-a4f185a68a95 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:49.439053 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee5619-2a9d-4bd4-a090-4a43c7dcfa67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:49.440091 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee5619-2a9d-4bd4-a090-4a43c7dcfa67 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:49.440673 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee5619-2a9d-4bd4-a090-4a43c7dcfa67 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:49.441081 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee5619-2a9d-4bd4-a090-4a43c7dcfa67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:49.442722 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ee5619-2a9d-4bd4-a090-4a43c7dcfa67 None admin] RBAC: Adding query filter params (role.id=ed7fec18968241efa76a0ee769d96da9, scope.project.id=0e408b6ba932474fb12e507cab566abd, user.id=af42430d72cf49eda0d60ad908dc340f) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:49.443295 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ee5619-2a9d-4bd4-a090-4a43c7dcfa67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:49.446197 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ee5619-2a9d-4bd4-a090-4a43c7dcfa67 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:49.457369 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:49.477338 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 135/266] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:07:49 2022] GET /identity/v3/role_assignments?user.id=af42430d72cf49eda0d60ad908dc340f&scope.project.id=0e408b6ba932474fb12e507cab566abd&role.id=ed7fec18968241efa76a0ee769d96da9 => generated 608 bytes in 421 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:49.494794 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c0f2c-b76f-49fd-9e1c-abf12033c740 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:49.558417 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c777bb30-3ac6-47f0-a03b-010e344b3fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:49.559475 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c777bb30-3ac6-47f0-a03b-010e344b3fd1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:49.559976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c777bb30-3ac6-47f0-a03b-010e344b3fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:49.560414 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c777bb30-3ac6-47f0-a03b-010e344b3fd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:49.655424 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c777bb30-3ac6-47f0-a03b-010e344b3fd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:49.657362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c777bb30-3ac6-47f0-a03b-010e344b3fd1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:49.776601 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:49.820209 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 132/267] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:07:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 19:07:49.832795 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee5619-2a9d-4bd4-a090-4a43c7dcfa67 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:49.945423 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941105d-3cb6-4b43-a138-0f7f69528cd8 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:49.947254 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941105d-3cb6-4b43-a138-0f7f69528cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:49.948118 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941105d-3cb6-4b43-a138-0f7f69528cd8 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:49.950213 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 133/268] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:07:49.960528 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789b98e-ce55-47e3-8fdd-a64bc5f0a139 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:49.961265 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789b98e-ce55-47e3-8fdd-a64bc5f0a139 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:49.961839 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789b98e-ce55-47e3-8fdd-a64bc5f0a139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:50.015541 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a789b98e-ce55-47e3-8fdd-a64bc5f0a139 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:50.017619 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a789b98e-ce55-47e3-8fdd-a64bc5f0a139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:50.046228 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee5619-2a9d-4bd4-a090-4a43c7dcfa67 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:50.120399 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f3b7b8-069e-4cac-96cc-44bc65319e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:50.121913 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3b7b8-069e-4cac-96cc-44bc65319e72 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:50.122425 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3b7b8-069e-4cac-96cc-44bc65319e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:50.122938 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3b7b8-069e-4cac-96cc-44bc65319e72 None admin] PATH_INFO: `/v3/projects/0e408b6ba932474fb12e507cab566abd` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:50.128189 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f3b7b8-069e-4cac-96cc-44bc65319e72 None admin] RBAC: Authorizing `identity:get_project(project_id=0e408b6ba932474fb12e507cab566abd)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:50.130735 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f3b7b8-069e-4cac-96cc-44bc65319e72 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:50.134341 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 136/269] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:07:49 2022] GET /identity/v3/projects/0e408b6ba932474fb12e507cab566abd => generated 302 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:50.136876 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5f2bf-a403-433e-a3e7-3f6e0a3cb86e None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:50.137379 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5f2bf-a403-433e-a3e7-3f6e0a3cb86e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:50.137740 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5f2bf-a403-433e-a3e7-3f6e0a3cb86e None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:50.143174 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 137/270] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:07:50.151063 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5f2bf-a403-433e-a3e7-3f6e0a3cb86e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:50.188714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 134/271] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:07:50.198012 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1a002-4e3d-4290-965c-073ad391208e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:50.198508 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1a002-4e3d-4290-965c-073ad391208e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:50.199014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1a002-4e3d-4290-965c-073ad391208e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:50.223638 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:50.238671 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-15d1a002-4e3d-4290-965c-073ad391208e None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:50.241497 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-15d1a002-4e3d-4290-965c-073ad391208e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:50.370686 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5f2bf-a403-433e-a3e7-3f6e0a3cb86e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:50.397245 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 135/272] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:07:50.399361 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1a002-4e3d-4290-965c-073ad391208e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:50.449023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07ab5a0-9f7f-4ac7-b07e-e5cb528fbc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:50.449699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ab5a0-9f7f-4ac7-b07e-e5cb528fbc81 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:50.449987 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ab5a0-9f7f-4ac7-b07e-e5cb528fbc81 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:50.450262 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ab5a0-9f7f-4ac7-b07e-e5cb528fbc81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:50.451139 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07ab5a0-9f7f-4ac7-b07e-e5cb528fbc81 None admin] RBAC: Adding query filter params (role.id=191e7e31dad148b7a0eb52d3afb5e50e, scope.project.id=0e408b6ba932474fb12e507cab566abd, user.id=af42430d72cf49eda0d60ad908dc340f) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:50.452551 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07ab5a0-9f7f-4ac7-b07e-e5cb528fbc81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:50.454278 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07ab5a0-9f7f-4ac7-b07e-e5cb528fbc81 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:50.462720 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 138/273] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:07:50 2022] GET /identity/v3/role_assignments?user.id=af42430d72cf49eda0d60ad908dc340f&scope.project.id=0e408b6ba932474fb12e507cab566abd&role.id=191e7e31dad148b7a0eb52d3afb5e50e => generated 608 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:07:50.467647 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07ab5a0-9f7f-4ac7-b07e-e5cb528fbc81 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:50.472931 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:50.647054 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1a002-4e3d-4290-965c-073ad391208e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:50.660143 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07ab5a0-9f7f-4ac7-b07e-e5cb528fbc81 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:50.663941 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:50.706928 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb0cdf5-9928-4539-806e-a11b2582b158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:50.707796 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0cdf5-9928-4539-806e-a11b2582b158 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:50.708079 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0cdf5-9928-4539-806e-a11b2582b158 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:50.708358 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0cdf5-9928-4539-806e-a11b2582b158 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:50.709809 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb0cdf5-9928-4539-806e-a11b2582b158 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:50.711422 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb0cdf5-9928-4539-806e-a11b2582b158 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:50.713280 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 139/274] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:07:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:50.714635 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952690b-fcd9-4272-86b4-bccf96a1f6ac None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:50.714959 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952690b-fcd9-4272-86b4-bccf96a1f6ac None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:50.719913 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952690b-fcd9-4272-86b4-bccf96a1f6ac None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:50.721868 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 140/275] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:07:50.724990 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952690b-fcd9-4272-86b4-bccf96a1f6ac None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:50.752241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e77f01-0b5b-4dbf-8e65-ccd4c4af03eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:50.753632 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e77f01-0b5b-4dbf-8e65-ccd4c4af03eb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:50.754140 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e77f01-0b5b-4dbf-8e65-ccd4c4af03eb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:50.754644 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e77f01-0b5b-4dbf-8e65-ccd4c4af03eb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:50.777957 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e77f01-0b5b-4dbf-8e65-ccd4c4af03eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:07:50.797767 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-32e77f01-0b5b-4dbf-8e65-ccd4c4af03eb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 19:07:50.797767 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:07:50.797767 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:07:50.797767 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:07:50.797767 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:07:50.797767 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:07:50.797767 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:07:50.797767 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:07:50.797767 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:07:50.797767 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:07:50.797767 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:07:50.797767 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:07:50.797767 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:07:50.797767 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:07:50.797767 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:07:50.797767 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:07:50.798755 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:07:50.798755 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:07:50.798755 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:07:50.798755 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:07:50.798755 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:07:50.798755 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:07:50.798755 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:07:50.798755 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:07:50.798755 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:07:50.798755 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:07:50.798755 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:07:50.798755 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:07:50.798755 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:07:50.798755 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:07:50.798755 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:07:50.798755 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:07:50.799831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:07:50.799831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:07:50.799831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:07:50.799831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:07:50.799831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:07:50.799831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:07:50.799831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:07:50.799831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:07:50.799831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:07:50.799831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:07:50.799831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 19:07:50.799831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:07:50.802039 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 136/276] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:07:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 403 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:07:50.804411 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028114e-7101-4fec-a79d-203915eed65f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:50.804906 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028114e-7101-4fec-a79d-203915eed65f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:50.805389 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028114e-7101-4fec-a79d-203915eed65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:50.837105 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b028114e-7101-4fec-a79d-203915eed65f None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:50.837430 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b028114e-7101-4fec-a79d-203915eed65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:50.877520 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952690b-fcd9-4272-86b4-bccf96a1f6ac None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:50.923755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5016ec20-a141-4e9c-b557-661a90745de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:50.924825 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016ec20-a141-4e9c-b557-661a90745de1 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:50.925260 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016ec20-a141-4e9c-b557-661a90745de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:50.926605 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016ec20-a141-4e9c-b557-661a90745de1 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:50.928005 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5016ec20-a141-4e9c-b557-661a90745de1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:50.930783 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5016ec20-a141-4e9c-b557-661a90745de1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:50.942650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 137/277] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:07:50.943543 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b028114e-7101-4fec-a79d-203915eed65f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:50.979412 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 141/278] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Jul 25 19:07:50 2022] POST /identity/v3/users => generated 293 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:07:50.987097 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:50.989674 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5016ec20-a141-4e9c-b557-661a90745de1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:51.028383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:51.163214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b028114e-7101-4fec-a79d-203915eed65f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:51.176622 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5016ec20-a141-4e9c-b557-661a90745de1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:51.220583 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8995d61f-250d-407b-89e1-efc2e9503a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:51.221458 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995d61f-250d-407b-89e1-efc2e9503a2c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:51.221747 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995d61f-250d-407b-89e1-efc2e9503a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:51.222069 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995d61f-250d-407b-89e1-efc2e9503a2c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:51.239771 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8995d61f-250d-407b-89e1-efc2e9503a2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:07:51.246803 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de585ca5-bd4f-444b-a2fd-469bdd449690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:51.248902 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de585ca5-bd4f-444b-a2fd-469bdd449690 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:51.249624 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de585ca5-bd4f-444b-a2fd-469bdd449690 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:51.250393 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de585ca5-bd4f-444b-a2fd-469bdd449690 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:51.252144 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de585ca5-bd4f-444b-a2fd-469bdd449690 None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:51.252939 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de585ca5-bd4f-444b-a2fd-469bdd449690 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:51.256006 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de585ca5-bd4f-444b-a2fd-469bdd449690 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:51.256224 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-8995d61f-250d-407b-89e1-efc2e9503a2c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 19:07:51.256224 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:07:51.256224 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:07:51.256224 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:07:51.256224 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:07:51.256224 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:07:51.256224 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:07:51.256224 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:07:51.256224 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:07:51.256224 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:07:51.256224 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:07:51.256224 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:07:51.256224 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:07:51.256224 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:07:51.256224 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:07:51.256224 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:07:51.257996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:07:51.257996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:07:51.257996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:07:51.257996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:07:51.257996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:07:51.257996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:07:51.257996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:07:51.257996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:07:51.257996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:07:51.257996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:07:51.257996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:07:51.257996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:07:51.257996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:07:51.257996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:07:51.257996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:07:51.257996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:07:51.259453 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:07:51.259453 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:07:51.259453 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:07:51.259453 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:07:51.259453 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:07:51.259453 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:07:51.259453 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:07:51.259453 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:07:51.259453 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:07:51.259453 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:07:51.259453 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 19:07:51.259453 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:07:51.267585 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 142/279] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:07:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:07:51.273926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8995d61f-250d-407b-89e1-efc2e9503a2c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:51.282157 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 138/280] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Jul 25 19:07:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:51.291727 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de585ca5-bd4f-444b-a2fd-469bdd449690 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:51.452987 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:51.500209 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:51.514476 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8995d61f-250d-407b-89e1-efc2e9503a2c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:51.539751 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de585ca5-bd4f-444b-a2fd-469bdd449690 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:51.618195 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ad65a-7e52-416f-8449-88be42e600fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:51.620124 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ad65a-7e52-416f-8449-88be42e600fd None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:51.620599 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ad65a-7e52-416f-8449-88be42e600fd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:51.621078 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ad65a-7e52-416f-8449-88be42e600fd None admin] PATH_INFO: `/v3/users/fcc6df3421cf4a58b26eb056c6a2fea3` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:51.652599 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c5860-3db6-4590-bb0d-5a2521f74eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:51.653687 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c5860-3db6-4590-bb0d-5a2521f74eaa None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:51.654161 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c5860-3db6-4590-bb0d-5a2521f74eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:51.655315 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c5860-3db6-4590-bb0d-5a2521f74eaa None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:51.656829 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c5860-3db6-4590-bb0d-5a2521f74eaa None admin] RBAC: Adding query filter params (name=reader) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:51.657353 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c5860-3db6-4590-bb0d-5a2521f74eaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:51.660459 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c5860-3db6-4590-bb0d-5a2521f74eaa None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:51.669548 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ad65a-7e52-416f-8449-88be42e600fd None admin] RBAC: Authorizing `identity:get_user(user_id=fcc6df3421cf4a58b26eb056c6a2fea3)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:51.673004 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ad65a-7e52-416f-8449-88be42e600fd None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:51.678480 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 139/281] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:07:51 2022] GET /identity/v3/users/fcc6df3421cf4a58b26eb056c6a2fea3 => generated 293 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:07:51.689566 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ad65a-7e52-416f-8449-88be42e600fd None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:51.699765 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 143/282] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Jul 25 19:07:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 426 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:07:51.711483 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c5860-3db6-4590-bb0d-5a2521f74eaa None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:51.975237 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c5860-3db6-4590-bb0d-5a2521f74eaa None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:52.055912 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd1b3b-c8f6-4a15-b81f-d2e741e11fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:52.056917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd1b3b-c8f6-4a15-b81f-d2e741e11fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:52.057205 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd1b3b-c8f6-4a15-b81f-d2e741e11fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:52.057538 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd1b3b-c8f6-4a15-b81f-d2e741e11fa5 None admin] PATH_INFO: `/v3/users/59d30a538a004005802bb5854af33511` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:52.060316 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd1b3b-c8f6-4a15-b81f-d2e741e11fa5 None admin] RBAC: Authorizing `identity:get_user(user_id=59d30a538a004005802bb5854af33511)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:52.064439 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd1b3b-c8f6-4a15-b81f-d2e741e11fa5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:52.067118 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 144/283] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:07:51 2022] GET /identity/v3/users/59d30a538a004005802bb5854af33511 => generated 297 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:52.070813 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92271f72-39a0-4513-9de2-9eaa1525ae5f None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:52.071319 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92271f72-39a0-4513-9de2-9eaa1525ae5f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:52.072211 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92271f72-39a0-4513-9de2-9eaa1525ae5f None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:52.076305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 145/284] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:07:52.077234 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ad65a-7e52-416f-8449-88be42e600fd None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:52.080472 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92271f72-39a0-4513-9de2-9eaa1525ae5f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:52.089141 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:52.130344 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e83a2-1ada-4dda-a507-636f7da812d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:52.131047 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e83a2-1ada-4dda-a507-636f7da812d4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:52.131332 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e83a2-1ada-4dda-a507-636f7da812d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:52.131649 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e83a2-1ada-4dda-a507-636f7da812d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:52.132572 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e83a2-1ada-4dda-a507-636f7da812d4 None admin] RBAC: Adding query filter params (role.id=c241bd9634e24bcbb1854c7621d19f81, user.id=fcc6df3421cf4a58b26eb056c6a2fea3, scope.system=all) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:52.133088 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e83a2-1ada-4dda-a507-636f7da812d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:52.135038 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e83a2-1ada-4dda-a507-636f7da812d4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:52.144068 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 140/285] 10.222.0.18 () {56 vars in 1308 bytes} [Mon Jul 25 19:07:51 2022] GET /identity/v3/role_assignments?user.id=fcc6df3421cf4a58b26eb056c6a2fea3&scope.system=all&role.id=c241bd9634e24bcbb1854c7621d19f81 => generated 226 bytes in 456 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:52.146345 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95d95a-0d51-49a6-95d6-831cd180ed14 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:52.147035 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95d95a-0d51-49a6-95d6-831cd180ed14 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:52.147295 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95d95a-0d51-49a6-95d6-831cd180ed14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:52.181258 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8b95d95a-0d51-49a6-95d6-831cd180ed14 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:52.183461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8b95d95a-0d51-49a6-95d6-831cd180ed14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:52.187389 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:52.339581 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 141/286] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:07:52.388092 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95d95a-0d51-49a6-95d6-831cd180ed14 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:52.401263 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92271f72-39a0-4513-9de2-9eaa1525ae5f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:52.497588 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0bdfe8-b7e3-47a4-bacd-31392b7e7b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:52.499726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bdfe8-b7e3-47a4-bacd-31392b7e7b20 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:52.500201 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bdfe8-b7e3-47a4-bacd-31392b7e7b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:52.500710 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bdfe8-b7e3-47a4-bacd-31392b7e7b20 None admin] PATH_INFO: `/v3/projects/0e408b6ba932474fb12e507cab566abd` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:52.502967 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0bdfe8-b7e3-47a4-bacd-31392b7e7b20 None admin] RBAC: Authorizing `identity:get_project(project_id=0e408b6ba932474fb12e507cab566abd)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:52.505654 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0bdfe8-b7e3-47a4-bacd-31392b7e7b20 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:52.509324 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 146/287] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:07:52 2022] GET /identity/v3/projects/0e408b6ba932474fb12e507cab566abd => generated 302 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:52.521821 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0bdfe8-b7e3-47a4-bacd-31392b7e7b20 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:52.667796 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:52.731555 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0bdfe8-b7e3-47a4-bacd-31392b7e7b20 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:52.782227 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e267cf-0e75-4caf-a36b-02e5656e2a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:52.783167 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e267cf-0e75-4caf-a36b-02e5656e2a64 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:52.783453 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e267cf-0e75-4caf-a36b-02e5656e2a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:52.783802 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e267cf-0e75-4caf-a36b-02e5656e2a64 None admin] PATH_INFO: `/v3/projects/0e408b6ba932474fb12e507cab566abd/users/59d30a538a004005802bb5854af33511/roles/6327898c72b044889755dec9746f54dd` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:52.786341 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e267cf-0e75-4caf-a36b-02e5656e2a64 None admin] RBAC: Authorizing `identity:create_grant(project_id=0e408b6ba932474fb12e507cab566abd, user_id=59d30a538a004005802bb5854af33511, role_id=6327898c72b044889755dec9746f54dd)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:52.787909 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e267cf-0e75-4caf-a36b-02e5656e2a64 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:52.807288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95d95a-0d51-49a6-95d6-831cd180ed14 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:52.808120 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 147/288] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Jul 25 19:07:52 2022] PUT /identity/v3/projects/0e408b6ba932474fb12e507cab566abd/users/59d30a538a004005802bb5854af33511/roles/6327898c72b044889755dec9746f54dd => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:07:52.902381 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c095f-a44e-460b-b125-4caaa4b463e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:52.904216 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c095f-a44e-460b-b125-4caaa4b463e4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:52.907920 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c095f-a44e-460b-b125-4caaa4b463e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:52.908687 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c095f-a44e-460b-b125-4caaa4b463e4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:52.931617 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:52.943669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5c095f-a44e-460b-b125-4caaa4b463e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:07:52.974573 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-3e5c095f-a44e-460b-b125-4caaa4b463e4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 19:07:52.974573 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:07:52.974573 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:07:52.974573 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:07:52.974573 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:07:52.974573 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:07:52.974573 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:07:52.974573 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:07:52.974573 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:07:52.974573 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:07:52.974573 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:07:52.974573 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:07:52.974573 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:07:52.974573 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:07:52.974573 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:07:52.974573 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:07:52.976598 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:07:52.976598 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:07:52.976598 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:07:52.976598 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:07:52.976598 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:07:52.976598 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:07:52.976598 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:07:52.976598 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:07:52.976598 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:07:52.976598 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:07:52.976598 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:07:52.976598 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:07:52.976598 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:07:52.976598 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:07:52.976598 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:07:52.976598 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:07:52.978324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:07:52.978324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:07:52.978324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:07:52.978324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:07:52.978324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:07:52.978324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:07:52.978324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:07:52.978324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:07:52.978324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:07:52.978324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:07:52.978324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 19:07:52.978324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:07:52.978324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 142/289] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:07:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 590 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:07:52.988042 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e267cf-0e75-4caf-a36b-02e5656e2a64 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:53.280395 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e267cf-0e75-4caf-a36b-02e5656e2a64 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:53.362625 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:53.378322 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac459b7a-c6ea-4596-9a5f-73db2ee4bf72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:53.382471 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac459b7a-c6ea-4596-9a5f-73db2ee4bf72 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:53.382940 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac459b7a-c6ea-4596-9a5f-73db2ee4bf72 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:53.383440 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac459b7a-c6ea-4596-9a5f-73db2ee4bf72 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:53.384973 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac459b7a-c6ea-4596-9a5f-73db2ee4bf72 None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:53.385518 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac459b7a-c6ea-4596-9a5f-73db2ee4bf72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:53.388227 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac459b7a-c6ea-4596-9a5f-73db2ee4bf72 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:53.408292 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 148/290] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Jul 25 19:07:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 421 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:53.417968 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c095f-a44e-460b-b125-4caaa4b463e4 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:53.686929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c095f-a44e-460b-b125-4caaa4b463e4 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:53.785378 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e896d-9b72-49d2-a9dd-fad3781de7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:53.786989 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e896d-9b72-49d2-a9dd-fad3781de7d2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:53.787552 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e896d-9b72-49d2-a9dd-fad3781de7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:53.788131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e896d-9b72-49d2-a9dd-fad3781de7d2 None admin] PATH_INFO: `/v3/users/f71d2f3680bd44519cb14fcb9545d89a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:53.792568 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8e896d-9b72-49d2-a9dd-fad3781de7d2 None admin] RBAC: Authorizing `identity:get_user(user_id=f71d2f3680bd44519cb14fcb9545d89a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:53.794486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8e896d-9b72-49d2-a9dd-fad3781de7d2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:53.797146 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 143/291] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:07:53 2022] GET /identity/v3/users/f71d2f3680bd44519cb14fcb9545d89a => generated 297 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:53.804253 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac459b7a-c6ea-4596-9a5f-73db2ee4bf72 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:54.174752 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac459b7a-c6ea-4596-9a5f-73db2ee4bf72 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:54.276331 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7cde4-2af4-480c-b4e3-3e5a435138b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:54.277031 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7cde4-2af4-480c-b4e3-3e5a435138b7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:54.277318 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7cde4-2af4-480c-b4e3-3e5a435138b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:54.277670 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7cde4-2af4-480c-b4e3-3e5a435138b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:54.450057 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:54.474525 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd7cde4-2af4-480c-b4e3-3e5a435138b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:54.477546 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd7cde4-2af4-480c-b4e3-3e5a435138b7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:54.687600 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 149/292] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:07:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 884 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 19:07:54.697085 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e896d-9b72-49d2-a9dd-fad3781de7d2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:54.848100 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:54.914106 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e896d-9b72-49d2-a9dd-fad3781de7d2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:54.976619 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8864e6-b826-4cd8-b381-23cd5278e376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:54.977532 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8864e6-b826-4cd8-b381-23cd5278e376 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:54.977866 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8864e6-b826-4cd8-b381-23cd5278e376 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:54.978154 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8864e6-b826-4cd8-b381-23cd5278e376 None admin] PATH_INFO: `/v3/projects/0e408b6ba932474fb12e507cab566abd` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:54.979681 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8864e6-b826-4cd8-b381-23cd5278e376 None admin] RBAC: Authorizing `identity:get_project(project_id=0e408b6ba932474fb12e507cab566abd)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:54.981316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8864e6-b826-4cd8-b381-23cd5278e376 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:54.984773 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 144/293] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:07:54 2022] GET /identity/v3/projects/0e408b6ba932474fb12e507cab566abd => generated 302 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:54.994686 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7cde4-2af4-480c-b4e3-3e5a435138b7 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:55.160929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9963f-3890-4b0d-85d6-dacd9bde0ba9 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:55.161164 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9963f-3890-4b0d-85d6-dacd9bde0ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:55.161483 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9963f-3890-4b0d-85d6-dacd9bde0ba9 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:55.163169 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 145/294] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:07:55.175059 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d028f97-fb9a-44ed-827f-2a13916c7718 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:55.175391 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d028f97-fb9a-44ed-827f-2a13916c7718 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:55.175667 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d028f97-fb9a-44ed-827f-2a13916c7718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:55.229198 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0d028f97-fb9a-44ed-827f-2a13916c7718 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:55.232228 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0d028f97-fb9a-44ed-827f-2a13916c7718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:55.236203 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:55.246558 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7cde4-2af4-480c-b4e3-3e5a435138b7 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:55.337789 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0dc07-8d32-418a-83e9-b68513429e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:55.338491 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0dc07-8d32-418a-83e9-b68513429e7b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:55.338784 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0dc07-8d32-418a-83e9-b68513429e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:55.339071 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0dc07-8d32-418a-83e9-b68513429e7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:55.339986 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f0dc07-8d32-418a-83e9-b68513429e7b None admin] RBAC: Adding query filter params (role.id=c241bd9634e24bcbb1854c7621d19f81, scope.project.id=0e408b6ba932474fb12e507cab566abd, user.id=f71d2f3680bd44519cb14fcb9545d89a) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:55.340310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f0dc07-8d32-418a-83e9-b68513429e7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:55.342805 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f0dc07-8d32-418a-83e9-b68513429e7b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:55.351594 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 150/295] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:07:54 2022] GET /identity/v3/role_assignments?user.id=f71d2f3680bd44519cb14fcb9545d89a&scope.project.id=0e408b6ba932474fb12e507cab566abd&role.id=c241bd9634e24bcbb1854c7621d19f81 => generated 608 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:07:55.439235 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 146/296] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:07:55.459163 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f88c52-97e3-4c2d-91f3-a5239c39dd27 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:55.459653 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f88c52-97e3-4c2d-91f3-a5239c39dd27 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:55.460131 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f88c52-97e3-4c2d-91f3-a5239c39dd27 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:55.463324 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 151/297] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:07:55.472829 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aab3ab-188a-47cb-b977-bebebe71e527 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:55.473269 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aab3ab-188a-47cb-b977-bebebe71e527 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:55.473703 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aab3ab-188a-47cb-b977-bebebe71e527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:55.479303 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f88c52-97e3-4c2d-91f3-a5239c39dd27 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:55.509188 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e8aab3ab-188a-47cb-b977-bebebe71e527 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:55.511595 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e8aab3ab-188a-47cb-b977-bebebe71e527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:55.704403 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:55.731464 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 147/298] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 19:07:55.784988 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aab3ab-188a-47cb-b977-bebebe71e527 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:55.868906 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f88c52-97e3-4c2d-91f3-a5239c39dd27 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:55.978131 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066fca37-4852-48b4-9ad8-9f88e3b81d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:55.979500 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066fca37-4852-48b4-9ad8-9f88e3b81d75 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:55.980073 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066fca37-4852-48b4-9ad8-9f88e3b81d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:55.980586 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066fca37-4852-48b4-9ad8-9f88e3b81d75 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:55.983958 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066fca37-4852-48b4-9ad8-9f88e3b81d75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:55.987077 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066fca37-4852-48b4-9ad8-9f88e3b81d75 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:55.990876 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 152/299] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:07:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 512 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:56.000257 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066fca37-4852-48b4-9ad8-9f88e3b81d75 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:56.066693 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:56.084321 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aab3ab-188a-47cb-b977-bebebe71e527 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:56.171504 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0848d6-43fd-4366-9c7f-05700a393b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:56.173275 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0848d6-43fd-4366-9c7f-05700a393b01 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:56.174276 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0848d6-43fd-4366-9c7f-05700a393b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:56.174276 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0848d6-43fd-4366-9c7f-05700a393b01 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:56.196845 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0848d6-43fd-4366-9c7f-05700a393b01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:07:56.213345 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-fb0848d6-43fd-4366-9c7f-05700a393b01 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 19:07:56.213345 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:07:56.213345 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:07:56.213345 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:07:56.213345 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:07:56.213345 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:07:56.213345 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:07:56.213345 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:07:56.213345 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:07:56.213345 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:07:56.213345 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:07:56.213345 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:07:56.213345 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:07:56.213345 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:07:56.213345 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:07:56.213345 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:07:56.215083 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:07:56.215083 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:07:56.215083 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:07:56.215083 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:07:56.215083 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:07:56.215083 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:07:56.215083 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:07:56.215083 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:07:56.215083 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:07:56.215083 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:07:56.215083 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:07:56.215083 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:07:56.215083 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:07:56.215083 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:07:56.215083 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:07:56.215083 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:07:56.223988 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:07:56.223988 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:07:56.223988 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:07:56.223988 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:07:56.223988 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:07:56.223988 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:07:56.223988 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:07:56.223988 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:07:56.223988 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:07:56.223988 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:07:56.223988 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 19:07:56.223988 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:07:56.223988 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 148/300] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:07:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 433 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:07:56.231664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0848d6-43fd-4366-9c7f-05700a393b01 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:56.245603 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066fca37-4852-48b4-9ad8-9f88e3b81d75 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:56.321257 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226241ae-af8c-44b5-94c1-6f30eb165f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:56.322480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226241ae-af8c-44b5-94c1-6f30eb165f4c None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:56.323794 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226241ae-af8c-44b5-94c1-6f30eb165f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:56.324359 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226241ae-af8c-44b5-94c1-6f30eb165f4c None admin] PATH_INFO: `/v3/groups` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:56.325695 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226241ae-af8c-44b5-94c1-6f30eb165f4c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:56.330266 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226241ae-af8c-44b5-94c1-6f30eb165f4c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:56.358959 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:56.365789 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 153/301] 10.222.0.18 () {60 vars in 1134 bytes} [Mon Jul 25 19:07:55 2022] POST /identity/v3/groups => generated 229 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:07:56.407010 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b39132-3fd0-47fa-bfbc-56a3d8c497c2 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:56.408103 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b39132-3fd0-47fa-bfbc-56a3d8c497c2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:56.408830 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b39132-3fd0-47fa-bfbc-56a3d8c497c2 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:56.412931 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 154/302] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:56 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:07:56.424113 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a1de58-aa9a-4c66-a56d-592a771d96c5 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:56.424988 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a1de58-aa9a-4c66-a56d-592a771d96c5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:56.425675 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a1de58-aa9a-4c66-a56d-592a771d96c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:56.465123 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c5a1de58-aa9a-4c66-a56d-592a771d96c5 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:56.469146 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c5a1de58-aa9a-4c66-a56d-592a771d96c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:56.480220 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0848d6-43fd-4366-9c7f-05700a393b01 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:56.555928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b7d98-0b39-4879-ad2f-b6b54f4ef960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:56.557746 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b7d98-0b39-4879-ad2f-b6b54f4ef960 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:56.558612 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b7d98-0b39-4879-ad2f-b6b54f4ef960 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:56.559363 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b7d98-0b39-4879-ad2f-b6b54f4ef960 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:56.561678 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53b7d98-0b39-4879-ad2f-b6b54f4ef960 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:56.562564 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53b7d98-0b39-4879-ad2f-b6b54f4ef960 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:56.565679 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53b7d98-0b39-4879-ad2f-b6b54f4ef960 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:56.590878 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 149/303] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Jul 25 19:07:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:56.593257 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b7d98-0b39-4879-ad2f-b6b54f4ef960 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:56.659795 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 155/304] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:07:56.698478 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a1de58-aa9a-4c66-a56d-592a771d96c5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:56.736819 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:56.805832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b7d98-0b39-4879-ad2f-b6b54f4ef960 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:56.856107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:56.884794 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76f9c9-3569-4fa2-ab26-a9e2fc1d2aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:56.885831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76f9c9-3569-4fa2-ab26-a9e2fc1d2aec None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:56.886122 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76f9c9-3569-4fa2-ab26-a9e2fc1d2aec None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:56.886469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76f9c9-3569-4fa2-ab26-a9e2fc1d2aec None admin] PATH_INFO: `/v3/users/8b2afa13f34942b49d705602cf493f86` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:56.915584 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a1de58-aa9a-4c66-a56d-592a771d96c5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:56.925022 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff76f9c9-3569-4fa2-ab26-a9e2fc1d2aec None admin] RBAC: Authorizing `identity:get_user(user_id=8b2afa13f34942b49d705602cf493f86)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:56.926893 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff76f9c9-3569-4fa2-ab26-a9e2fc1d2aec None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:56.929711 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 150/305] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:07:56 2022] GET /identity/v3/users/8b2afa13f34942b49d705602cf493f86 => generated 293 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:56.938453 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76f9c9-3569-4fa2-ab26-a9e2fc1d2aec None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:56.991475 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c986e5-0ae2-4fc7-870e-c317213bfe27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:56.993105 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c986e5-0ae2-4fc7-870e-c317213bfe27 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:56.994187 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c986e5-0ae2-4fc7-870e-c317213bfe27 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:56.994790 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c986e5-0ae2-4fc7-870e-c317213bfe27 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:57.017505 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c986e5-0ae2-4fc7-870e-c317213bfe27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:07:57.034351 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-48c986e5-0ae2-4fc7-870e-c317213bfe27 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 19:07:57.034351 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:07:57.034351 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:07:57.034351 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:07:57.034351 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:07:57.034351 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:07:57.034351 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:07:57.034351 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:07:57.034351 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:07:57.034351 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:07:57.034351 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:07:57.034351 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:07:57.034351 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:07:57.034351 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:07:57.034351 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:07:57.034351 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:07:57.037041 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:07:57.037041 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:07:57.037041 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:07:57.037041 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:07:57.037041 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:07:57.037041 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:07:57.037041 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:07:57.037041 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:07:57.037041 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:07:57.037041 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:07:57.037041 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:07:57.037041 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:07:57.037041 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:07:57.037041 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:07:57.037041 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:07:57.037041 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:07:57.040145 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:07:57.040145 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:07:57.040145 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:07:57.040145 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:07:57.040145 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:07:57.040145 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:07:57.040145 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:07:57.040145 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:07:57.040145 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:07:57.040145 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:07:57.040145 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 19:07:57.040145 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:07:57.048082 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 156/306] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:07:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:07:57.049945 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c986e5-0ae2-4fc7-870e-c317213bfe27 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:57.128436 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76f9c9-3569-4fa2-ab26-a9e2fc1d2aec None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:57.201340 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe6fae-11ee-4c94-b2db-0ce207482829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:57.202829 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe6fae-11ee-4c94-b2db-0ce207482829 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:57.203344 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe6fae-11ee-4c94-b2db-0ce207482829 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:57.203880 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe6fae-11ee-4c94-b2db-0ce207482829 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:57.205606 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fe6fae-11ee-4c94-b2db-0ce207482829 None admin] RBAC: Adding query filter params (role.id=6327898c72b044889755dec9746f54dd, user.id=8b2afa13f34942b49d705602cf493f86, scope.system=all) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:57.206181 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fe6fae-11ee-4c94-b2db-0ce207482829 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:57.209240 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fe6fae-11ee-4c94-b2db-0ce207482829 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:57.212038 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:57.224944 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 151/307] 10.222.0.18 () {56 vars in 1308 bytes} [Mon Jul 25 19:07:56 2022] GET /identity/v3/role_assignments?user.id=8b2afa13f34942b49d705602cf493f86&scope.system=all&role.id=6327898c72b044889755dec9746f54dd => generated 226 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:57.264690 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c986e5-0ae2-4fc7-870e-c317213bfe27 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:57.324655 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8713c-4238-44b8-979e-1c68329baa01 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:57.325885 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8713c-4238-44b8-979e-1c68329baa01 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:57.327425 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8713c-4238-44b8-979e-1c68329baa01 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:57.330895 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 152/308] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:07:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:07:57.340817 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8404f-10d9-4537-aca1-166cef4bb52a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:57.341312 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8404f-10d9-4537-aca1-166cef4bb52a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:57.342725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8404f-10d9-4537-aca1-166cef4bb52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:57.346630 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74213b07-0117-4a83-92da-1da24d3b5e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:57.348257 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74213b07-0117-4a83-92da-1da24d3b5e7c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:57.348959 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74213b07-0117-4a83-92da-1da24d3b5e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:57.349454 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74213b07-0117-4a83-92da-1da24d3b5e7c None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:57.350543 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74213b07-0117-4a83-92da-1da24d3b5e7c None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:57.351329 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74213b07-0117-4a83-92da-1da24d3b5e7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:57.353498 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74213b07-0117-4a83-92da-1da24d3b5e7c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:57.367910 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 157/309] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Jul 25 19:07:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:57.378597 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74213b07-0117-4a83-92da-1da24d3b5e7c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:57.381190 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ffd8404f-10d9-4537-aca1-166cef4bb52a None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:07:57.383622 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ffd8404f-10d9-4537-aca1-166cef4bb52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:07:57.456503 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:57.543763 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 153/310] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:07:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 19:07:57.586579 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd8404f-10d9-4537-aca1-166cef4bb52a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:57.598203 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74213b07-0117-4a83-92da-1da24d3b5e7c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:57.660181 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5375e4d-3d3d-492d-88fb-f0dbe2ba5fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:57.661094 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5375e4d-3d3d-492d-88fb-f0dbe2ba5fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:57.661389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5375e4d-3d3d-492d-88fb-f0dbe2ba5fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:57.661864 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5375e4d-3d3d-492d-88fb-f0dbe2ba5fc0 None admin] PATH_INFO: `/v3/users/fcc6df3421cf4a58b26eb056c6a2fea3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:57.663810 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5375e4d-3d3d-492d-88fb-f0dbe2ba5fc0 None admin] RBAC: Authorizing `identity:get_user(user_id=fcc6df3421cf4a58b26eb056c6a2fea3)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:57.665674 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5375e4d-3d3d-492d-88fb-f0dbe2ba5fc0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:57.674054 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 158/311] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:07:57 2022] GET /identity/v3/users/fcc6df3421cf4a58b26eb056c6a2fea3 => generated 293 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:07:57.675822 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5375e4d-3d3d-492d-88fb-f0dbe2ba5fc0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:57.688305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:57.842546 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5375e4d-3d3d-492d-88fb-f0dbe2ba5fc0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:57.848210 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd8404f-10d9-4537-aca1-166cef4bb52a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:57.905308 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80211d50-3fc0-4df1-a98d-b54bb1dbba0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:57.906371 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80211d50-3fc0-4df1-a98d-b54bb1dbba0d None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:57.906820 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80211d50-3fc0-4df1-a98d-b54bb1dbba0d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:57.907205 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80211d50-3fc0-4df1-a98d-b54bb1dbba0d None admin] PATH_INFO: `/v3/system/users/fcc6df3421cf4a58b26eb056c6a2fea3/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:57.909688 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80211d50-3fc0-4df1-a98d-b54bb1dbba0d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=fcc6df3421cf4a58b26eb056c6a2fea3, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:57.912297 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80211d50-3fc0-4df1-a98d-b54bb1dbba0d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:57.928960 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 159/312] 10.222.0.18 () {58 vars in 1334 bytes} [Mon Jul 25 19:07:57 2022] PUT /identity/v3/system/users/fcc6df3421cf4a58b26eb056c6a2fea3/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:07:57.929330 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06447a70-b783-4595-bb70-77e1a871079d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:57.930873 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06447a70-b783-4595-bb70-77e1a871079d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:57.931676 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06447a70-b783-4595-bb70-77e1a871079d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:57.932203 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06447a70-b783-4595-bb70-77e1a871079d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:57.957374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06447a70-b783-4595-bb70-77e1a871079d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:07:57.974557 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-06447a70-b783-4595-bb70-77e1a871079d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 19:07:57.974557 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:07:57.974557 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:07:57.974557 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:07:57.974557 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:07:57.974557 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:07:57.974557 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:07:57.974557 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:07:57.974557 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:07:57.974557 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:07:57.974557 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:07:57.974557 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:07:57.974557 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:07:57.974557 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:07:57.974557 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:07:57.974557 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:07:57.975768 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:07:57.975768 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:07:57.975768 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:07:57.975768 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:07:57.975768 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:07:57.975768 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:07:57.975768 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:07:57.975768 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:07:57.975768 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:07:57.975768 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:07:57.975768 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:07:57.975768 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:07:57.975768 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:07:57.975768 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:07:57.975768 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:07:57.975768 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:07:57.976678 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:07:57.976678 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:07:57.976678 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:07:57.976678 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:07:57.976678 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:07:57.976678 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:07:57.976678 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:07:57.976678 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:07:57.976678 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:07:57.976678 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:07:57.976678 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 19:07:57.976678 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:07:57.977754 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 154/313] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:07:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:07:57.984796 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80211d50-3fc0-4df1-a98d-b54bb1dbba0d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:58.092754 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:58.160842 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80211d50-3fc0-4df1-a98d-b54bb1dbba0d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:58.240095 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b90fe19-2878-46a7-b1f7-2c0072b8bbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:58.241340 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90fe19-2878-46a7-b1f7-2c0072b8bbfe None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:58.241860 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90fe19-2878-46a7-b1f7-2c0072b8bbfe None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:58.242378 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90fe19-2878-46a7-b1f7-2c0072b8bbfe None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:58.243943 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b90fe19-2878-46a7-b1f7-2c0072b8bbfe None admin] RBAC: Adding query filter params (name=reader) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:58.244512 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b90fe19-2878-46a7-b1f7-2c0072b8bbfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:58.247249 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b90fe19-2878-46a7-b1f7-2c0072b8bbfe None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:58.269989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 160/314] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Jul 25 19:07:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:07:58.277719 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06447a70-b783-4595-bb70-77e1a871079d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:58.332028 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:58.476622 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06447a70-b783-4595-bb70-77e1a871079d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:58.550419 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7933e5b6-60f3-4da9-b1ac-707698f9cb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:58.552705 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933e5b6-60f3-4da9-b1ac-707698f9cb9c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:58.553572 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933e5b6-60f3-4da9-b1ac-707698f9cb9c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:58.554348 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933e5b6-60f3-4da9-b1ac-707698f9cb9c None admin] PATH_INFO: `/v3/users/59d30a538a004005802bb5854af33511` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:58.557192 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7933e5b6-60f3-4da9-b1ac-707698f9cb9c None admin] RBAC: Authorizing `identity:get_user(user_id=59d30a538a004005802bb5854af33511)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:58.560463 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7933e5b6-60f3-4da9-b1ac-707698f9cb9c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:58.565522 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 155/315] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:07:58 2022] GET /identity/v3/users/59d30a538a004005802bb5854af33511 => generated 297 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:58.572023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b90fe19-2878-46a7-b1f7-2c0072b8bbfe None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:58.800385 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b90fe19-2878-46a7-b1f7-2c0072b8bbfe None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:58.861888 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:58.871456 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bddac8-f98d-424c-b01f-a28a6877760a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:58.873920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bddac8-f98d-424c-b01f-a28a6877760a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:58.875249 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bddac8-f98d-424c-b01f-a28a6877760a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:58.875249 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bddac8-f98d-424c-b01f-a28a6877760a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:59.028315 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bddac8-f98d-424c-b01f-a28a6877760a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:59.030853 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bddac8-f98d-424c-b01f-a28a6877760a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:59.269192 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 161/316] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:07:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 698 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 19:07:59.279352 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7933e5b6-60f3-4da9-b1ac-707698f9cb9c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:59.487185 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:59.502793 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7933e5b6-60f3-4da9-b1ac-707698f9cb9c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:59.576575 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f85df-c613-4e53-9b4f-07b86492728b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:59.578162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f85df-c613-4e53-9b4f-07b86492728b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:59.578646 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f85df-c613-4e53-9b4f-07b86492728b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:59.579130 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f85df-c613-4e53-9b4f-07b86492728b None admin] PATH_INFO: `/v3/projects/0e408b6ba932474fb12e507cab566abd` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:59.581682 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f85df-c613-4e53-9b4f-07b86492728b None admin] RBAC: Authorizing `identity:get_project(project_id=0e408b6ba932474fb12e507cab566abd)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:59.584594 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f85df-c613-4e53-9b4f-07b86492728b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:59.588579 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 156/317] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:07:59 2022] GET /identity/v3/projects/0e408b6ba932474fb12e507cab566abd => generated 302 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:07:59.599720 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bddac8-f98d-424c-b01f-a28a6877760a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:07:59.670044 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:07:59.829466 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bddac8-f98d-424c-b01f-a28a6877760a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:07:59.909843 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab4fee1-33bf-43bf-8d15-817d836a5c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:07:59.911701 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab4fee1-33bf-43bf-8d15-817d836a5c99 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:07:59.912748 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab4fee1-33bf-43bf-8d15-817d836a5c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:07:59.913563 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab4fee1-33bf-43bf-8d15-817d836a5c99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:07:59.915675 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab4fee1-33bf-43bf-8d15-817d836a5c99 None admin] RBAC: Adding query filter params (role.id=6327898c72b044889755dec9746f54dd, scope.project.id=0e408b6ba932474fb12e507cab566abd, user.id=59d30a538a004005802bb5854af33511) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:07:59.916664 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab4fee1-33bf-43bf-8d15-817d836a5c99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:07:59.920003 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab4fee1-33bf-43bf-8d15-817d836a5c99 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:07:59.934562 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 162/318] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:07:59 2022] GET /identity/v3/role_assignments?user.id=59d30a538a004005802bb5854af33511&scope.project.id=0e408b6ba932474fb12e507cab566abd&role.id=6327898c72b044889755dec9746f54dd => generated 608 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:00.203900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2146fd25-4c32-453a-be57-3dcda9aae0c3 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:00.204405 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2146fd25-4c32-453a-be57-3dcda9aae0c3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:00.204925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2146fd25-4c32-453a-be57-3dcda9aae0c3 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:00.207785 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 157/319] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:08:00.219137 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ebb40-31e3-441e-ba87-20f179e22490 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:00.219684 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ebb40-31e3-441e-ba87-20f179e22490 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:00.220183 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ebb40-31e3-441e-ba87-20f179e22490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:00.255669 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-077ebb40-31e3-441e-ba87-20f179e22490 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:00.257896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-077ebb40-31e3-441e-ba87-20f179e22490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:00.414106 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 163/320] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:08:00.459929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2146fd25-4c32-453a-be57-3dcda9aae0c3 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:00.659784 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2146fd25-4c32-453a-be57-3dcda9aae0c3 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:00.689462 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aee25e-01cb-4708-8a11-9a89cd56dd4c None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:00.689962 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aee25e-01cb-4708-8a11-9a89cd56dd4c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:00.690486 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aee25e-01cb-4708-8a11-9a89cd56dd4c None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:00.693181 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 164/321] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:08:00.704475 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac53c2d-369d-4241-b79b-cbd56157e3d3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:00.704962 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac53c2d-369d-4241-b79b-cbd56157e3d3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:00.705431 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac53c2d-369d-4241-b79b-cbd56157e3d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:00.727618 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58750dae-2e56-455b-ac2a-41ed3117af23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:00.728955 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58750dae-2e56-455b-ac2a-41ed3117af23 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:00.729484 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58750dae-2e56-455b-ac2a-41ed3117af23 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:00.729980 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58750dae-2e56-455b-ac2a-41ed3117af23 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:00.732976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58750dae-2e56-455b-ac2a-41ed3117af23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:00.736007 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58750dae-2e56-455b-ac2a-41ed3117af23 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:00.739139 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-aac53c2d-369d-4241-b79b-cbd56157e3d3 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:00.739820 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 158/322] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:08:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:08:00.741391 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-aac53c2d-369d-4241-b79b-cbd56157e3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:00.749721 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58750dae-2e56-455b-ac2a-41ed3117af23 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:00.786330 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:00.818429 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:00.884507 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 165/323] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:08:00.926004 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac53c2d-369d-4241-b79b-cbd56157e3d3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:00.974490 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58750dae-2e56-455b-ac2a-41ed3117af23 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:01.053302 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d946558-2aa9-4e95-80d5-e574792f1c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:01.054397 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d946558-2aa9-4e95-80d5-e574792f1c52 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:01.054959 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d946558-2aa9-4e95-80d5-e574792f1c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:01.055438 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d946558-2aa9-4e95-80d5-e574792f1c52 None admin] PATH_INFO: `/v3/groups` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:01.056815 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d946558-2aa9-4e95-80d5-e574792f1c52 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:01.059695 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d946558-2aa9-4e95-80d5-e574792f1c52 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:01.093179 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 159/324] 10.222.0.18 () {60 vars in 1134 bytes} [Mon Jul 25 19:08:00 2022] POST /identity/v3/groups => generated 226 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:08:01.161684 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac53c2d-369d-4241-b79b-cbd56157e3d3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:01.226761 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1fd557-1d65-4a70-beb9-f87f30cc297d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:01.227740 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fd557-1d65-4a70-beb9-f87f30cc297d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:01.228081 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fd557-1d65-4a70-beb9-f87f30cc297d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:01.228399 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fd557-1d65-4a70-beb9-f87f30cc297d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:01.244136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1fd557-1d65-4a70-beb9-f87f30cc297d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:08:01.254585 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-1c1fd557-1d65-4a70-beb9-f87f30cc297d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 19:08:01.254585 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:01.254585 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:01.254585 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:01.254585 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:01.254585 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:01.254585 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:01.254585 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:01.254585 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:01.254585 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:01.254585 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:01.254585 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:01.254585 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:08:01.254585 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:08:01.254585 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:08:01.254585 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:08:01.269632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:08:01.269632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:08:01.269632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:08:01.269632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:08:01.269632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:01.269632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:01.269632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:01.269632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:01.269632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:01.269632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:01.269632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:01.269632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:01.269632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:01.269632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:01.269632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:01.269632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:01.271420 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:01.271420 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:01.271420 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:01.271420 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:01.271420 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:08:01.271420 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:08:01.271420 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:08:01.271420 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:08:01.271420 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:08:01.271420 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:08:01.271420 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 19:08:01.271420 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:01.271420 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 166/325] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:08:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:01.277245 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d946558-2aa9-4e95-80d5-e574792f1c52 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:01.309339 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21329e85-3578-4926-9078-e0053f6a251b None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:01.309757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21329e85-3578-4926-9078-e0053f6a251b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:01.310134 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21329e85-3578-4926-9078-e0053f6a251b None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:01.312216 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 167/326] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:08:01.319813 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a7299-8c3d-4e41-9163-690e865ec91a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:01.320748 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a7299-8c3d-4e41-9163-690e865ec91a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:01.320748 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a7299-8c3d-4e41-9163-690e865ec91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:01.358984 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1b6a7299-8c3d-4e41-9163-690e865ec91a None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:01.360525 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1b6a7299-8c3d-4e41-9163-690e865ec91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:01.383385 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:01.481676 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:01.502036 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d946558-2aa9-4e95-80d5-e574792f1c52 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:01.515317 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 168/327] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 19:08:01.557463 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a7299-8c3d-4e41-9163-690e865ec91a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:01.575715 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482e4c5-34b5-4a4c-8828-9dbf24e38c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:01.577467 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482e4c5-34b5-4a4c-8828-9dbf24e38c62 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:01.577755 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482e4c5-34b5-4a4c-8828-9dbf24e38c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:01.578177 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482e4c5-34b5-4a4c-8828-9dbf24e38c62 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:01.579462 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b482e4c5-34b5-4a4c-8828-9dbf24e38c62 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:01.579976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b482e4c5-34b5-4a4c-8828-9dbf24e38c62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:01.582568 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b482e4c5-34b5-4a4c-8828-9dbf24e38c62 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:01.600586 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 160/328] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Jul 25 19:08:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:01.612116 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482e4c5-34b5-4a4c-8828-9dbf24e38c62 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:01.742005 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a7299-8c3d-4e41-9163-690e865ec91a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:01.817708 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f03c7-b1f7-4465-8ed8-06a44e2be868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:01.819919 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f03c7-b1f7-4465-8ed8-06a44e2be868 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:01.821407 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f03c7-b1f7-4465-8ed8-06a44e2be868 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:01.823159 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f03c7-b1f7-4465-8ed8-06a44e2be868 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:01.836504 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482e4c5-34b5-4a4c-8828-9dbf24e38c62 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:01.854210 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335f03c7-b1f7-4465-8ed8-06a44e2be868 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:08:01.877001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-335f03c7-b1f7-4465-8ed8-06a44e2be868 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 19:08:01.877001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:01.877001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:01.877001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:01.877001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:01.877001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:01.877001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:01.877001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:01.877001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:01.877001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:01.877001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:01.877001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:01.877001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:08:01.877001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:08:01.877001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:08:01.877001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:08:01.878718 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:08:01.878718 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:08:01.878718 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:08:01.878718 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:08:01.878718 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:01.878718 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:01.878718 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:01.878718 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:01.878718 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:01.878718 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:01.878718 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:01.878718 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:01.878718 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:01.878718 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:01.878718 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:01.878718 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:01.880291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:01.880291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:01.880291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:01.880291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:01.880291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:08:01.880291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:08:01.880291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:08:01.880291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:08:01.880291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:08:01.880291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:08:01.880291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 19:08:01.880291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:01.880291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 169/329] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:08:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:08:01.891887 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f03c7-b1f7-4465-8ed8-06a44e2be868 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:01.922380 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:01.924467 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ccf9e-d64c-474f-b95b-418a1343d72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:01.926529 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ccf9e-d64c-474f-b95b-418a1343d72e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:01.927399 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ccf9e-d64c-474f-b95b-418a1343d72e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:01.928494 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ccf9e-d64c-474f-b95b-418a1343d72e None admin] PATH_INFO: `/v3/users/8b2afa13f34942b49d705602cf493f86` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:01.931914 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ccf9e-d64c-474f-b95b-418a1343d72e None admin] RBAC: Authorizing `identity:get_user(user_id=8b2afa13f34942b49d705602cf493f86)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:01.934922 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ccf9e-d64c-474f-b95b-418a1343d72e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:01.940252 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 161/330] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:08:01 2022] GET /identity/v3/users/8b2afa13f34942b49d705602cf493f86 => generated 293 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:01.951972 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ccf9e-d64c-474f-b95b-418a1343d72e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:02.012317 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:02.113909 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f03c7-b1f7-4465-8ed8-06a44e2be868 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:02.143272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ccf9e-d64c-474f-b95b-418a1343d72e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:02.160849 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64028044-719c-4b82-9698-e25a55606c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:02.161778 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64028044-719c-4b82-9698-e25a55606c3c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:02.162069 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64028044-719c-4b82-9698-e25a55606c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:02.162872 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64028044-719c-4b82-9698-e25a55606c3c None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:02.165012 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64028044-719c-4b82-9698-e25a55606c3c None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:02.165352 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64028044-719c-4b82-9698-e25a55606c3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:02.167213 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64028044-719c-4b82-9698-e25a55606c3c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:02.181985 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 170/331] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Jul 25 19:08:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:08:02.190125 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64028044-719c-4b82-9698-e25a55606c3c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:02.194589 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd68e9-2eff-4b85-8c4e-6a43be19e5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:02.196268 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd68e9-2eff-4b85-8c4e-6a43be19e5da None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:02.196855 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd68e9-2eff-4b85-8c4e-6a43be19e5da None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:02.197438 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd68e9-2eff-4b85-8c4e-6a43be19e5da None admin] PATH_INFO: `/v3/system/users/8b2afa13f34942b49d705602cf493f86/roles/6327898c72b044889755dec9746f54dd` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:02.199718 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd68e9-2eff-4b85-8c4e-6a43be19e5da None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8b2afa13f34942b49d705602cf493f86, role_id=6327898c72b044889755dec9746f54dd)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:02.201727 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd68e9-2eff-4b85-8c4e-6a43be19e5da None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:02.214732 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 162/332] 10.222.0.18 () {58 vars in 1334 bytes} [Mon Jul 25 19:08:01 2022] PUT /identity/v3/system/users/8b2afa13f34942b49d705602cf493f86/roles/6327898c72b044889755dec9746f54dd => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:08:02.304021 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:02.376877 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64028044-719c-4b82-9698-e25a55606c3c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:02.442843 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47ea19-0177-44c9-b744-63fc1306b999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:02.444624 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47ea19-0177-44c9-b744-63fc1306b999 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:02.445123 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47ea19-0177-44c9-b744-63fc1306b999 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:02.445531 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47ea19-0177-44c9-b744-63fc1306b999 None admin] PATH_INFO: `/v3/users/fcc6df3421cf4a58b26eb056c6a2fea3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:02.448060 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c47ea19-0177-44c9-b744-63fc1306b999 None admin] RBAC: Authorizing `identity:get_user(user_id=fcc6df3421cf4a58b26eb056c6a2fea3)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:02.451598 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c47ea19-0177-44c9-b744-63fc1306b999 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:02.455083 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 171/333] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:08:02 2022] GET /identity/v3/users/fcc6df3421cf4a58b26eb056c6a2fea3 => generated 293 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:02.461800 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd68e9-2eff-4b85-8c4e-6a43be19e5da None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:02.667736 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:02.711328 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd68e9-2eff-4b85-8c4e-6a43be19e5da None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:02.790718 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b75b14-8c2a-4eaf-94d2-5343d17a9459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:02.792123 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b75b14-8c2a-4eaf-94d2-5343d17a9459 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:02.792619 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b75b14-8c2a-4eaf-94d2-5343d17a9459 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:02.793200 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b75b14-8c2a-4eaf-94d2-5343d17a9459 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:02.794357 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b75b14-8c2a-4eaf-94d2-5343d17a9459 None admin] RBAC: Adding query filter params (role.id=c241bd9634e24bcbb1854c7621d19f81, user.id=fcc6df3421cf4a58b26eb056c6a2fea3, scope.system=all) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:02.795025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b75b14-8c2a-4eaf-94d2-5343d17a9459 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:02.797415 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b75b14-8c2a-4eaf-94d2-5343d17a9459 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:02.821921 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 163/334] 10.222.0.18 () {56 vars in 1308 bytes} [Mon Jul 25 19:08:02 2022] GET /identity/v3/role_assignments?user.id=fcc6df3421cf4a58b26eb056c6a2fea3&scope.system=all&role.id=c241bd9634e24bcbb1854c7621d19f81 => generated 510 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:08:05.557546 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e753efb-f027-4771-84b8-7e69e1483e38 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:05.558155 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e753efb-f027-4771-84b8-7e69e1483e38 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:05.558155 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e753efb-f027-4771-84b8-7e69e1483e38 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:05.559701 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 172/335] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:08:05.566315 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4e06e-b2c5-434f-805b-fe592d177998 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:05.567290 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4e06e-b2c5-434f-805b-fe592d177998 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:05.567846 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4e06e-b2c5-434f-805b-fe592d177998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:05.600793 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-baf4e06e-b2c5-434f-805b-fe592d177998 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:05.602757 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-baf4e06e-b2c5-434f-805b-fe592d177998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:05.698347 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 164/336] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:08:05.773154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e753efb-f027-4771-84b8-7e69e1483e38 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:05.937724 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e753efb-f027-4771-84b8-7e69e1483e38 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:05.985486 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e06fd-419d-485a-b599-0cb16dbd776b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:05.986621 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e06fd-419d-485a-b599-0cb16dbd776b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:05.987086 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e06fd-419d-485a-b599-0cb16dbd776b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:05.987601 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e06fd-419d-485a-b599-0cb16dbd776b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:05.994791 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:06.005857 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47e06fd-419d-485a-b599-0cb16dbd776b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:08:06.015226 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d47e06fd-419d-485a-b599-0cb16dbd776b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 19:08:06.015226 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:06.015226 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:06.015226 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:06.015226 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:06.015226 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:06.015226 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:06.015226 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:06.015226 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:06.015226 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:06.015226 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:06.015226 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:06.015226 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:08:06.015226 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:08:06.015226 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:08:06.015226 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:08:06.017651 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:08:06.017651 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:08:06.017651 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:08:06.017651 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:08:06.017651 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:06.017651 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:06.017651 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:06.017651 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:06.017651 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:06.017651 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:06.017651 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:06.017651 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:06.017651 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:06.017651 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:06.017651 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:06.017651 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:06.019917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:06.019917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:06.019917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:06.019917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:06.019917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:08:06.019917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:08:06.019917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:08:06.019917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:08:06.019917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:08:06.019917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:08:06.019917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 19:08:06.019917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:06.019917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 173/337] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:08:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:08:06.027112 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4e06e-b2c5-434f-805b-fe592d177998 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:06.048371 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae2cc3-b257-4347-9632-592ef14b8a3b None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:06.048797 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae2cc3-b257-4347-9632-592ef14b8a3b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:06.049033 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae2cc3-b257-4347-9632-592ef14b8a3b None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:06.050687 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 174/338] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:08:06.057715 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3f080-df42-44c5-b656-7224216540e6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:06.058190 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3f080-df42-44c5-b656-7224216540e6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:06.058676 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3f080-df42-44c5-b656-7224216540e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:06.125214 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-55e3f080-df42-44c5-b656-7224216540e6 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:06.127338 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-55e3f080-df42-44c5-b656-7224216540e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:06.303803 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:06.330147 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 175/339] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 19:08:06.359256 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4e06e-b2c5-434f-805b-fe592d177998 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:06.413769 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ea6a1-523f-4b77-b0f8-01a7acd420a0 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:06.414453 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ea6a1-523f-4b77-b0f8-01a7acd420a0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:06.415024 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ea6a1-523f-4b77-b0f8-01a7acd420a0 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:06.416959 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 176/340] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:08:06.418050 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ea6a1-523f-4b77-b0f8-01a7acd420a0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:06.436672 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e12848-c9b4-43de-a489-65ed90d24ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:06.437985 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e12848-c9b4-43de-a489-65ed90d24ed5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:06.438507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e12848-c9b4-43de-a489-65ed90d24ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:06.439005 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e12848-c9b4-43de-a489-65ed90d24ed5 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:06.440719 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e12848-c9b4-43de-a489-65ed90d24ed5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:06.441300 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e12848-c9b4-43de-a489-65ed90d24ed5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:06.444313 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e12848-c9b4-43de-a489-65ed90d24ed5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:06.472488 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 165/341] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Jul 25 19:08:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 446 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:08:06.475613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7358ac-8f83-4d8f-8ef1-9fdfe7563efb None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:06.476399 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7358ac-8f83-4d8f-8ef1-9fdfe7563efb None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:06.477296 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7358ac-8f83-4d8f-8ef1-9fdfe7563efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:06.523983 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7f7358ac-8f83-4d8f-8ef1-9fdfe7563efb None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:06.526778 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7f7358ac-8f83-4d8f-8ef1-9fdfe7563efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:06.582491 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ea6a1-523f-4b77-b0f8-01a7acd420a0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:06.668108 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:06.700353 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa3e082-eb4e-4f01-b97d-261ecc5bd05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:06.704378 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3e082-eb4e-4f01-b97d-261ecc5bd05c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:06.705490 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3e082-eb4e-4f01-b97d-261ecc5bd05c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:06.707291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3e082-eb4e-4f01-b97d-261ecc5bd05c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:06.754081 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa3e082-eb4e-4f01-b97d-261ecc5bd05c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:08:06.772376 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-eaa3e082-eb4e-4f01-b97d-261ecc5bd05c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 19:08:06.772376 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:06.772376 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:06.772376 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:06.772376 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:06.772376 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:06.772376 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:06.772376 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:06.772376 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:06.772376 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:06.772376 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:06.772376 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:06.772376 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:08:06.772376 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:08:06.772376 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:08:06.772376 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:08:06.774493 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:08:06.774493 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:08:06.774493 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:08:06.774493 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:08:06.774493 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:06.774493 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:06.774493 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:06.774493 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:06.774493 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:06.774493 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:06.774493 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:06.774493 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:06.774493 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:06.774493 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:06.774493 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:06.774493 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:06.776459 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:06.776459 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:06.776459 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:06.776459 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:06.776459 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:08:06.776459 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:08:06.776459 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:08:06.776459 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:08:06.776459 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:08:06.776459 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:08:06.776459 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 19:08:06.776459 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:06.780934 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 177/342] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Jul 25 19:08:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:06.782697 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa3e082-eb4e-4f01-b97d-261ecc5bd05c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:06.784766 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 166/343] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:08:06.809735 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7358ac-8f83-4d8f-8ef1-9fdfe7563efb None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:06.976362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:06.984876 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa3e082-eb4e-4f01-b97d-261ecc5bd05c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:07.038091 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7358ac-8f83-4d8f-8ef1-9fdfe7563efb None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:07.069161 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f9d7c9-c680-47b4-af59-69816c8e6532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:07.069833 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9d7c9-c680-47b4-af59-69816c8e6532 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:07.070426 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9d7c9-c680-47b4-af59-69816c8e6532 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:07.070829 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9d7c9-c680-47b4-af59-69816c8e6532 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:07.098668 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87988ac5-c25d-4355-8a66-6de759bf5d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:07.099880 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87988ac5-c25d-4355-8a66-6de759bf5d5e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:07.100357 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87988ac5-c25d-4355-8a66-6de759bf5d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:07.100840 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87988ac5-c25d-4355-8a66-6de759bf5d5e None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:07.102944 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87988ac5-c25d-4355-8a66-6de759bf5d5e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:07.102944 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87988ac5-c25d-4355-8a66-6de759bf5d5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:07.105589 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87988ac5-c25d-4355-8a66-6de759bf5d5e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:07.118336 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 167/344] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:08:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:07.120306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664cb81-9516-48a1-91d2-dfe3649cacbb None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:07.120844 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664cb81-9516-48a1-91d2-dfe3649cacbb None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:07.121127 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664cb81-9516-48a1-91d2-dfe3649cacbb None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:07.123573 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 168/345] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:08:07.124789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664cb81-9516-48a1-91d2-dfe3649cacbb None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:07.161254 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:07.199450 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f9d7c9-c680-47b4-af59-69816c8e6532 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:07.203105 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f9d7c9-c680-47b4-af59-69816c8e6532 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:07.300813 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664cb81-9516-48a1-91d2-dfe3649cacbb None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:07.336587 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:07.349509 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcf5a15-46f8-47a9-8e02-fd18bc9d6a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:07.350456 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf5a15-46f8-47a9-8e02-fd18bc9d6a28 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:07.350746 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf5a15-46f8-47a9-8e02-fd18bc9d6a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:07.351027 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf5a15-46f8-47a9-8e02-fd18bc9d6a28 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:07.366559 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcf5a15-46f8-47a9-8e02-fd18bc9d6a28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:08:07.375756 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-5fcf5a15-46f8-47a9-8e02-fd18bc9d6a28 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 19:08:07.375756 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:07.375756 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:07.375756 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:07.375756 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:07.375756 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:07.375756 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:07.375756 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:07.375756 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:07.375756 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:07.375756 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:07.375756 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:07.375756 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:08:07.375756 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:08:07.375756 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:08:07.375756 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:08:07.377222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:08:07.377222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:08:07.377222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:08:07.377222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:08:07.377222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:07.377222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:07.377222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:07.377222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:07.377222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:07.377222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:07.377222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:07.377222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:07.377222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:07.377222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:07.377222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:07.377222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:07.378758 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:07.378758 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:07.378758 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:07.378758 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:07.378758 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:08:07.378758 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:08:07.378758 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:08:07.378758 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:08:07.378758 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:08:07.378758 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:08:07.378758 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 19:08:07.378758 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:07.378758 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 169/346] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:08:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:07.378758 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ee6b9-b8bb-4f2d-a387-bc062832ddd2 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:07.384295 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ee6b9-b8bb-4f2d-a387-bc062832ddd2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:07.384295 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ee6b9-b8bb-4f2d-a387-bc062832ddd2 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:07.384295 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 170/347] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:08:07.384295 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ee6b9-b8bb-4f2d-a387-bc062832ddd2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:07.405566 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 178/348] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:08:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 19:08:07.408197 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb01a97-98e5-43d7-81d0-002e3645379c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:07.408793 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb01a97-98e5-43d7-81d0-002e3645379c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:07.409380 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb01a97-98e5-43d7-81d0-002e3645379c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:07.437382 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3fb01a97-98e5-43d7-81d0-002e3645379c None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:07.438893 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3fb01a97-98e5-43d7-81d0-002e3645379c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:07.556901 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ee6b9-b8bb-4f2d-a387-bc062832ddd2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:07.567129 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 179/349] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:08:07.568900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb01a97-98e5-43d7-81d0-002e3645379c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:07.611950 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:07.629386 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851becc-4f86-40ce-abd0-5db2f2831732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:07.631565 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851becc-4f86-40ce-abd0-5db2f2831732 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:07.631565 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851becc-4f86-40ce-abd0-5db2f2831732 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:07.631565 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851becc-4f86-40ce-abd0-5db2f2831732 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:07.729104 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:07.748841 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb01a97-98e5-43d7-81d0-002e3645379c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:07.758409 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851becc-4f86-40ce-abd0-5db2f2831732 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:07.758690 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851becc-4f86-40ce-abd0-5db2f2831732 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:07.808383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a7b0a-bdc9-46c2-83cb-c8257a660155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:07.808984 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a7b0a-bdc9-46c2-83cb-c8257a660155 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:07.809290 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a7b0a-bdc9-46c2-83cb-c8257a660155 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:07.809718 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a7b0a-bdc9-46c2-83cb-c8257a660155 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:07.810859 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a7b0a-bdc9-46c2-83cb-c8257a660155 None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:07.811351 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a7b0a-bdc9-46c2-83cb-c8257a660155 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:07.813132 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a7b0a-bdc9-46c2-83cb-c8257a660155 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:07.828043 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 180/350] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Jul 25 19:08:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:07.829399 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5030d86-e953-4bf5-aff4-144a59c0487f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:07.830480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5030d86-e953-4bf5-aff4-144a59c0487f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:07.830480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5030d86-e953-4bf5-aff4-144a59c0487f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:07.852652 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f5030d86-e953-4bf5-aff4-144a59c0487f None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:07.853927 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f5030d86-e953-4bf5-aff4-144a59c0487f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:07.954774 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 171/351] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:08:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 19:08:07.955911 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851becc-4f86-40ce-abd0-5db2f2831732 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:08.003678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 181/352] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:08:08.004755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5030d86-e953-4bf5-aff4-144a59c0487f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:08.023442 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:08.124395 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:08.145633 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851becc-4f86-40ce-abd0-5db2f2831732 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:08.179186 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5030d86-e953-4bf5-aff4-144a59c0487f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:08.206657 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf6724-b76a-4ee7-97fb-ca02431783cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:08.208605 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf6724-b76a-4ee7-97fb-ca02431783cb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:08.209137 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf6724-b76a-4ee7-97fb-ca02431783cb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:08.209633 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf6724-b76a-4ee7-97fb-ca02431783cb None admin] PATH_INFO: `/v3/projects/09f6174e875f4cf58a2eb5019ef11189` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:08.214413 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf6724-b76a-4ee7-97fb-ca02431783cb None admin] RBAC: Authorizing `identity:get_project(project_id=09f6174e875f4cf58a2eb5019ef11189)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:08.217994 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf6724-b76a-4ee7-97fb-ca02431783cb None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:08.222144 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 172/353] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:08:07 2022] GET /identity/v3/projects/09f6174e875f4cf58a2eb5019ef11189 => generated 298 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:08.225505 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55a1a7-22d1-4774-bde6-46acdf0baa9f None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:08.226059 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55a1a7-22d1-4774-bde6-46acdf0baa9f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:08.226555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55a1a7-22d1-4774-bde6-46acdf0baa9f None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:08.230321 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 173/354] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:08:08.231652 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef55a1a7-22d1-4774-bde6-46acdf0baa9f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:08.253259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5ddbc-511d-43bd-a033-b5cad38fc428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:08.254300 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5ddbc-511d-43bd-a033-b5cad38fc428 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:08.254726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5ddbc-511d-43bd-a033-b5cad38fc428 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:08.255135 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5ddbc-511d-43bd-a033-b5cad38fc428 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:08.276330 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b5ddbc-511d-43bd-a033-b5cad38fc428 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:08:08.286101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-64b5ddbc-511d-43bd-a033-b5cad38fc428 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 19:08:08.286101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:08.286101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:08.286101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:08.286101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:08.286101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:08.286101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:08.286101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:08.286101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:08.286101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:08.286101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:08.286101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:08.286101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:08:08.286101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:08:08.286101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:08:08.286101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:08:08.287161 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:08:08.287161 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:08:08.287161 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:08:08.287161 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:08:08.287161 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:08.287161 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:08.287161 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:08.287161 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:08.287161 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:08.287161 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:08.287161 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:08.287161 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:08.287161 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:08.287161 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:08.287161 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:08.287161 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:08.287994 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:08.287994 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:08.287994 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:08.287994 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:08.287994 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:08:08.287994 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:08:08.287994 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:08:08.287994 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:08:08.287994 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:08:08.287994 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:08:08.287994 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 19:08:08.287994 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:08.290416 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 182/355] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:08:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:08.291284 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5ddbc-511d-43bd-a033-b5cad38fc428 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:08.430783 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:08.435757 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef55a1a7-22d1-4774-bde6-46acdf0baa9f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:08.504884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5ddbc-511d-43bd-a033-b5cad38fc428 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:08.506472 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a05dc5-df7a-47bc-98a8-cd37c6b958e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:08.507953 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a05dc5-df7a-47bc-98a8-cd37c6b958e7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:08.509841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a05dc5-df7a-47bc-98a8-cd37c6b958e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:08.510520 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a05dc5-df7a-47bc-98a8-cd37c6b958e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:08.564162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:08.575271 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d263f49-d2c8-43e4-ba53-352f7c8c4327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:08.578023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d263f49-d2c8-43e4-ba53-352f7c8c4327 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:08.578023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d263f49-d2c8-43e4-ba53-352f7c8c4327 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:08.578238 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d263f49-d2c8-43e4-ba53-352f7c8c4327 None admin] PATH_INFO: `/v3/projects/09f6174e875f4cf58a2eb5019ef11189` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:08.580870 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d263f49-d2c8-43e4-ba53-352f7c8c4327 None admin] RBAC: Authorizing `identity:get_project(project_id=09f6174e875f4cf58a2eb5019ef11189)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:08.583472 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d263f49-d2c8-43e4-ba53-352f7c8c4327 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:08.586475 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 183/356] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:08:08 2022] GET /identity/v3/projects/09f6174e875f4cf58a2eb5019ef11189 => generated 298 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:08.587744 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d263f49-d2c8-43e4-ba53-352f7c8c4327 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:08.648863 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a05dc5-df7a-47bc-98a8-cd37c6b958e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:08.650813 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a05dc5-df7a-47bc-98a8-cd37c6b958e7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:08.773892 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d263f49-d2c8-43e4-ba53-352f7c8c4327 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:08.830214 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e83153-4ef3-4f3c-819a-e45974055f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:08.831692 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e83153-4ef3-4f3c-819a-e45974055f8a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:08.832165 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e83153-4ef3-4f3c-819a-e45974055f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:08.832667 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e83153-4ef3-4f3c-819a-e45974055f8a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:08.840482 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:08.847641 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 174/357] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:08:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 617 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 19:08:08.848820 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a05dc5-df7a-47bc-98a8-cd37c6b958e7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:08.856301 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e83153-4ef3-4f3c-819a-e45974055f8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:08:08.871137 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d3e83153-4ef3-4f3c-819a-e45974055f8a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 19:08:08.871137 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:08.871137 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:08.871137 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:08.871137 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:08.871137 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:08.871137 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:08.871137 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:08.871137 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:08.871137 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:08.871137 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:08.871137 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:08.871137 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:08:08.871137 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:08:08.871137 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:08:08.871137 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:08:08.874448 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:08:08.874448 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:08:08.874448 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:08:08.874448 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:08:08.874448 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:08.874448 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:08.874448 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:08.874448 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:08.874448 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:08.874448 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:08.874448 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:08.874448 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:08.874448 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:08.874448 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:08.874448 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:08.874448 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:08.877657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:08.877657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:08.877657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:08.877657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:08.877657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:08:08.877657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:08:08.877657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:08:08.877657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:08:08.877657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:08:08.877657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:08:08.877657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 19:08:08.877657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:08.877657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 184/358] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Jul 25 19:08:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:08.877657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514cf255-b3f0-4dc3-baf7-d4e4697c3740 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:08.881224 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514cf255-b3f0-4dc3-baf7-d4e4697c3740 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:08.881224 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514cf255-b3f0-4dc3-baf7-d4e4697c3740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:08.905136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-514cf255-b3f0-4dc3-baf7-d4e4697c3740 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:08.907123 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-514cf255-b3f0-4dc3-baf7-d4e4697c3740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:08.961268 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:09.016074 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 185/359] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:08:09.017382 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514cf255-b3f0-4dc3-baf7-d4e4697c3740 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:09.039267 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a05dc5-df7a-47bc-98a8-cd37c6b958e7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:09.096106 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec21d3-38a4-4e20-ab90-1e88da48f554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:09.097482 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec21d3-38a4-4e20-ab90-1e88da48f554 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:09.097937 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec21d3-38a4-4e20-ab90-1e88da48f554 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:09.098380 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec21d3-38a4-4e20-ab90-1e88da48f554 None admin] PATH_INFO: `/v3/groups/7c8589271cc74ab88f7862a00a1915cb` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:09.109865 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ec21d3-38a4-4e20-ab90-1e88da48f554 None admin] RBAC: Authorizing `identity:get_group(group_id=7c8589271cc74ab88f7862a00a1915cb)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:09.112046 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ec21d3-38a4-4e20-ab90-1e88da48f554 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:09.114797 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 175/360] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Jul 25 19:08:08 2022] GET /identity/v3/groups/7c8589271cc74ab88f7862a00a1915cb => generated 226 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:09.116108 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec21d3-38a4-4e20-ab90-1e88da48f554 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:09.205600 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514cf255-b3f0-4dc3-baf7-d4e4697c3740 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:09.242424 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:09.255568 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f73eee-32ac-4f88-b1ac-81c10a4e48d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:09.256298 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f73eee-32ac-4f88-b1ac-81c10a4e48d1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:09.256627 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f73eee-32ac-4f88-b1ac-81c10a4e48d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:09.256971 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f73eee-32ac-4f88-b1ac-81c10a4e48d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:09.257758 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f73eee-32ac-4f88-b1ac-81c10a4e48d1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:09.258066 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f73eee-32ac-4f88-b1ac-81c10a4e48d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:09.261647 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f73eee-32ac-4f88-b1ac-81c10a4e48d1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:09.272030 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 186/361] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Jul 25 19:08:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:09.273157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f73eee-32ac-4f88-b1ac-81c10a4e48d1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:09.285613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec21d3-38a4-4e20-ab90-1e88da48f554 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:09.325801 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:09.352595 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b92d4-62ce-4c41-ba44-b8bb20a57f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:09.354091 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b92d4-62ce-4c41-ba44-b8bb20a57f5f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:09.354580 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b92d4-62ce-4c41-ba44-b8bb20a57f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:09.355069 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b92d4-62ce-4c41-ba44-b8bb20a57f5f None admin] PATH_INFO: `/v3/groups/7c8589271cc74ab88f7862a00a1915cb` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:09.357887 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5b92d4-62ce-4c41-ba44-b8bb20a57f5f None admin] RBAC: Authorizing `identity:get_group(group_id=7c8589271cc74ab88f7862a00a1915cb)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:09.360739 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5b92d4-62ce-4c41-ba44-b8bb20a57f5f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:09.364177 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 176/362] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Jul 25 19:08:09 2022] GET /identity/v3/groups/7c8589271cc74ab88f7862a00a1915cb => generated 226 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:09.365692 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b92d4-62ce-4c41-ba44-b8bb20a57f5f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:09.403251 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f73eee-32ac-4f88-b1ac-81c10a4e48d1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:09.448153 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a3652-3e14-4489-b6f0-422c2b6f634a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:09.449049 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a3652-3e14-4489-b6f0-422c2b6f634a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:09.449342 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a3652-3e14-4489-b6f0-422c2b6f634a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:09.449636 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a3652-3e14-4489-b6f0-422c2b6f634a None admin] PATH_INFO: `/v3/projects/0e408b6ba932474fb12e507cab566abd` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:09.451100 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7a3652-3e14-4489-b6f0-422c2b6f634a None admin] RBAC: Authorizing `identity:get_project(project_id=0e408b6ba932474fb12e507cab566abd)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:09.452754 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7a3652-3e14-4489-b6f0-422c2b6f634a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:09.454585 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 187/363] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:08:09 2022] GET /identity/v3/projects/0e408b6ba932474fb12e507cab566abd => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:09.456801 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a3652-3e14-4489-b6f0-422c2b6f634a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:09.533797 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b92d4-62ce-4c41-ba44-b8bb20a57f5f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:09.537781 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:09.601086 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a3652-3e14-4489-b6f0-422c2b6f634a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:09.603479 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca2bf8-a251-4d17-9fc9-2166f4ba143c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:09.604378 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca2bf8-a251-4d17-9fc9-2166f4ba143c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:09.604837 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca2bf8-a251-4d17-9fc9-2166f4ba143c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:09.605368 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca2bf8-a251-4d17-9fc9-2166f4ba143c None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:09.606470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca2bf8-a251-4d17-9fc9-2166f4ba143c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:09.607059 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca2bf8-a251-4d17-9fc9-2166f4ba143c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:09.609572 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca2bf8-a251-4d17-9fc9-2166f4ba143c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:09.622942 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 177/364] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:08:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:09.624000 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca2bf8-a251-4d17-9fc9-2166f4ba143c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:09.651805 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc13f3d3-dca6-461f-b3dc-337116499343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:09.652813 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13f3d3-dca6-461f-b3dc-337116499343 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:09.653187 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13f3d3-dca6-461f-b3dc-337116499343 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:09.653590 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13f3d3-dca6-461f-b3dc-337116499343 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:09.679821 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc13f3d3-dca6-461f-b3dc-337116499343 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:08:09.694770 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-dc13f3d3-dca6-461f-b3dc-337116499343 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:08:09.694770 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:09.694770 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:09.694770 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:09.694770 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:09.694770 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:09.694770 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:09.694770 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:09.694770 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:09.694770 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:09.694770 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:09.694770 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:09.694770 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:08:09.694770 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:08:09.694770 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:08:09.694770 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:08:09.695973 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:08:09.695973 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:08:09.695973 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:08:09.695973 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:08:09.695973 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:09.695973 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:09.695973 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:09.695973 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:09.695973 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:09.695973 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:09.695973 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:09.695973 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:09.695973 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:09.695973 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:09.695973 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:09.695973 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:09.696922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:09.696922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:09.696922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:09.696922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:09.696922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:08:09.696922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:08:09.696922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:08:09.696922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:08:09.696922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:08:09.696922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:08:09.696922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:08:09.696922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:09.696922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 188/365] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Jul 25 19:08:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:09.699088 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc13f3d3-dca6-461f-b3dc-337116499343 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:09.708872 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:09.784622 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca2bf8-a251-4d17-9fc9-2166f4ba143c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:09.828841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c1fdce-61b8-42c5-870f-41a1a03cbec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:09.829689 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1fdce-61b8-42c5-870f-41a1a03cbec9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:09.830100 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1fdce-61b8-42c5-870f-41a1a03cbec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:09.830527 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1fdce-61b8-42c5-870f-41a1a03cbec9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:09.831602 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c1fdce-61b8-42c5-870f-41a1a03cbec9 None admin] RBAC: Adding query filter params (group.id=7c8589271cc74ab88f7862a00a1915cb, role.id=c241bd9634e24bcbb1854c7621d19f81, scope.project.id=09f6174e875f4cf58a2eb5019ef11189) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:09.832046 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c1fdce-61b8-42c5-870f-41a1a03cbec9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:09.833761 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c1fdce-61b8-42c5-870f-41a1a03cbec9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:09.843492 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 178/366] 10.222.0.18 () {56 vars in 1376 bytes} [Mon Jul 25 19:08:09 2022] GET /identity/v3/role_assignments?group.id=7c8589271cc74ab88f7862a00a1915cb&scope.project.id=09f6174e875f4cf58a2eb5019ef11189&role.id=c241bd9634e24bcbb1854c7621d19f81 => generated 260 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:09.844985 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c1fdce-61b8-42c5-870f-41a1a03cbec9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:09.865130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc13f3d3-dca6-461f-b3dc-337116499343 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:09.893311 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:09.929666 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341da4ab-08a2-491c-a684-971c98875c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:09.930890 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341da4ab-08a2-491c-a684-971c98875c69 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:09.931333 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341da4ab-08a2-491c-a684-971c98875c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:09.931774 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341da4ab-08a2-491c-a684-971c98875c69 None admin] PATH_INFO: `/v3/users/8b2afa13f34942b49d705602cf493f86` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:09.933411 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341da4ab-08a2-491c-a684-971c98875c69 None admin] RBAC: Authorizing `identity:get_user(user_id=8b2afa13f34942b49d705602cf493f86)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:09.935740 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341da4ab-08a2-491c-a684-971c98875c69 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:09.945874 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 189/367] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:08:09 2022] GET /identity/v3/users/8b2afa13f34942b49d705602cf493f86 => generated 293 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:09.945874 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341da4ab-08a2-491c-a684-971c98875c69 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:10.050836 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c1fdce-61b8-42c5-870f-41a1a03cbec9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:10.068818 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:10.095707 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3ad71-980a-4642-b5f5-9cbf79f22d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:10.096386 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3ad71-980a-4642-b5f5-9cbf79f22d5f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:10.096671 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3ad71-980a-4642-b5f5-9cbf79f22d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:10.096948 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3ad71-980a-4642-b5f5-9cbf79f22d5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:10.097789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3ad71-980a-4642-b5f5-9cbf79f22d5f None admin] RBAC: Adding query filter params (group.id=7c8589271cc74ab88f7862a00a1915cb, role.id=191e7e31dad148b7a0eb52d3afb5e50e, scope.project.id=09f6174e875f4cf58a2eb5019ef11189) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:10.098102 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3ad71-980a-4642-b5f5-9cbf79f22d5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:10.099633 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d3ad71-980a-4642-b5f5-9cbf79f22d5f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:10.115716 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 179/368] 10.222.0.18 () {56 vars in 1376 bytes} [Mon Jul 25 19:08:09 2022] GET /identity/v3/role_assignments?group.id=7c8589271cc74ab88f7862a00a1915cb&scope.project.id=09f6174e875f4cf58a2eb5019ef11189&role.id=191e7e31dad148b7a0eb52d3afb5e50e => generated 260 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:10.115716 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3ad71-980a-4642-b5f5-9cbf79f22d5f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:10.157176 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341da4ab-08a2-491c-a684-971c98875c69 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:10.233531 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9213379-31ba-4456-8435-dbcabcb1b6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:10.235303 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9213379-31ba-4456-8435-dbcabcb1b6f9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:10.235984 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9213379-31ba-4456-8435-dbcabcb1b6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:10.236609 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9213379-31ba-4456-8435-dbcabcb1b6f9 None admin] PATH_INFO: `/v3/groups/7c8589271cc74ab88f7862a00a1915cb` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:10.238991 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9213379-31ba-4456-8435-dbcabcb1b6f9 None admin] RBAC: Authorizing `identity:get_group(group_id=7c8589271cc74ab88f7862a00a1915cb)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:10.241968 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9213379-31ba-4456-8435-dbcabcb1b6f9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:10.246073 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 190/369] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Jul 25 19:08:09 2022] GET /identity/v3/groups/7c8589271cc74ab88f7862a00a1915cb => generated 226 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:10.247655 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9213379-31ba-4456-8435-dbcabcb1b6f9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:10.313015 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3ad71-980a-4642-b5f5-9cbf79f22d5f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:10.358812 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46340c4c-8e22-4607-af4e-bb64347269d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:10.360027 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46340c4c-8e22-4607-af4e-bb64347269d4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:10.360545 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46340c4c-8e22-4607-af4e-bb64347269d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:10.361062 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46340c4c-8e22-4607-af4e-bb64347269d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:10.363426 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:10.438507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:10.479266 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46340c4c-8e22-4607-af4e-bb64347269d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:10.480810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46340c4c-8e22-4607-af4e-bb64347269d4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:10.494178 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9213379-31ba-4456-8435-dbcabcb1b6f9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:10.578063 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b273a7c5-dd6c-412c-ab43-6b556607a896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:10.578991 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273a7c5-dd6c-412c-ab43-6b556607a896 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:10.579387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273a7c5-dd6c-412c-ab43-6b556607a896 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:10.579844 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273a7c5-dd6c-412c-ab43-6b556607a896 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:10.580819 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b273a7c5-dd6c-412c-ab43-6b556607a896 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:10.581136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b273a7c5-dd6c-412c-ab43-6b556607a896 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:10.582655 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b273a7c5-dd6c-412c-ab43-6b556607a896 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:10.597809 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 191/370] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:08:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:10.599170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b273a7c5-dd6c-412c-ab43-6b556607a896 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:10.664621 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 180/371] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:08:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 19:08:10.665535 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46340c4c-8e22-4607-af4e-bb64347269d4 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:10.824734 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b273a7c5-dd6c-412c-ab43-6b556607a896 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:10.857855 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:10.865320 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:10.909268 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910f2d1-83bd-4877-ae8a-9a3e741b9172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:10.911284 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910f2d1-83bd-4877-ae8a-9a3e741b9172 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:10.912078 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910f2d1-83bd-4877-ae8a-9a3e741b9172 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:10.912765 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910f2d1-83bd-4877-ae8a-9a3e741b9172 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:10.915015 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4910f2d1-83bd-4877-ae8a-9a3e741b9172 None admin] RBAC: Adding query filter params (role.id=6327898c72b044889755dec9746f54dd, user.id=8b2afa13f34942b49d705602cf493f86, scope.system=all) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:10.915836 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4910f2d1-83bd-4877-ae8a-9a3e741b9172 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:10.916941 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46340c4c-8e22-4607-af4e-bb64347269d4 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:10.919416 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4910f2d1-83bd-4877-ae8a-9a3e741b9172 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:10.945999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 192/372] 10.222.0.18 () {56 vars in 1308 bytes} [Mon Jul 25 19:08:10 2022] GET /identity/v3/role_assignments?user.id=8b2afa13f34942b49d705602cf493f86&scope.system=all&role.id=6327898c72b044889755dec9746f54dd => generated 510 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:10.953649 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910f2d1-83bd-4877-ae8a-9a3e741b9172 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:11.012117 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8212ba-c752-44cb-894e-478ba31ccbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:11.013695 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8212ba-c752-44cb-894e-478ba31ccbfd None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:11.014183 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8212ba-c752-44cb-894e-478ba31ccbfd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:11.014648 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8212ba-c752-44cb-894e-478ba31ccbfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:11.017076 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8212ba-c752-44cb-894e-478ba31ccbfd None admin] RBAC: Adding query filter params (group.id=7c8589271cc74ab88f7862a00a1915cb, role.id=c241bd9634e24bcbb1854c7621d19f81, scope.project.id=0e408b6ba932474fb12e507cab566abd) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:11.017594 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8212ba-c752-44cb-894e-478ba31ccbfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:11.020152 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8212ba-c752-44cb-894e-478ba31ccbfd None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:11.031608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 181/373] 10.222.0.18 () {56 vars in 1376 bytes} [Mon Jul 25 19:08:10 2022] GET /identity/v3/role_assignments?group.id=7c8589271cc74ab88f7862a00a1915cb&scope.project.id=0e408b6ba932474fb12e507cab566abd&role.id=c241bd9634e24bcbb1854c7621d19f81 => generated 260 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:11.033065 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8212ba-c752-44cb-894e-478ba31ccbfd None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:11.290351 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8212ba-c752-44cb-894e-478ba31ccbfd None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:11.382078 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4154c9-58b9-452d-a17d-9c136f610cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:11.382999 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4154c9-58b9-452d-a17d-9c136f610cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:11.383281 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4154c9-58b9-452d-a17d-9c136f610cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:11.383588 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4154c9-58b9-452d-a17d-9c136f610cb6 None admin] PATH_INFO: `/v3/projects/0e408b6ba932474fb12e507cab566abd` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:11.385414 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4154c9-58b9-452d-a17d-9c136f610cb6 None admin] RBAC: Authorizing `identity:get_project(project_id=0e408b6ba932474fb12e507cab566abd)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:11.387320 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4154c9-58b9-452d-a17d-9c136f610cb6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:11.390065 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 182/374] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:08:11 2022] GET /identity/v3/projects/0e408b6ba932474fb12e507cab566abd => generated 302 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:11.400585 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4154c9-58b9-452d-a17d-9c136f610cb6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:11.423151 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910f2d1-83bd-4877-ae8a-9a3e741b9172 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:11.453815 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:11.532374 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ecbee5-2b7e-46b8-af62-1a2cd5ea2ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:11.533496 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ecbee5-2b7e-46b8-af62-1a2cd5ea2ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:11.533939 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ecbee5-2b7e-46b8-af62-1a2cd5ea2ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:11.534398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ecbee5-2b7e-46b8-af62-1a2cd5ea2ae9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:11.639895 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:11.722969 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4154c9-58b9-452d-a17d-9c136f610cb6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:11.757475 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ecbee5-2b7e-46b8-af62-1a2cd5ea2ae9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:11.759761 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ecbee5-2b7e-46b8-af62-1a2cd5ea2ae9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:11.804848 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7df7c-ec00-402d-9b15-e3b32845d185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:11.805900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7df7c-ec00-402d-9b15-e3b32845d185 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:11.806387 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7df7c-ec00-402d-9b15-e3b32845d185 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:11.806826 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7df7c-ec00-402d-9b15-e3b32845d185 None admin] PATH_INFO: `/v3/groups/7c8589271cc74ab88f7862a00a1915cb` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:11.810106 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b7df7c-ec00-402d-9b15-e3b32845d185 None admin] RBAC: Authorizing `identity:get_group(group_id=7c8589271cc74ab88f7862a00a1915cb)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:11.811884 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b7df7c-ec00-402d-9b15-e3b32845d185 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:11.814128 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 183/375] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Jul 25 19:08:11 2022] GET /identity/v3/groups/7c8589271cc74ab88f7862a00a1915cb => generated 226 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:08:11.826650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7df7c-ec00-402d-9b15-e3b32845d185 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:11.995860 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 193/376] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:08:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 1043 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 19:08:12.016796 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ecbee5-2b7e-46b8-af62-1a2cd5ea2ae9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:12.106919 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:12.116244 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:12.178016 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7df7c-ec00-402d-9b15-e3b32845d185 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:12.185815 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ecbee5-2b7e-46b8-af62-1a2cd5ea2ae9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:12.245184 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c491d-809c-4a92-adc2-8449e83cce91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:12.246037 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c491d-809c-4a92-adc2-8449e83cce91 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:12.246291 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c491d-809c-4a92-adc2-8449e83cce91 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:12.246535 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c491d-809c-4a92-adc2-8449e83cce91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:12.247439 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c491d-809c-4a92-adc2-8449e83cce91 None admin] RBAC: Adding query filter params (group.id=7c8589271cc74ab88f7862a00a1915cb, role.id=191e7e31dad148b7a0eb52d3afb5e50e, scope.project.id=0e408b6ba932474fb12e507cab566abd) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:12.247996 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c491d-809c-4a92-adc2-8449e83cce91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:12.249596 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c491d-809c-4a92-adc2-8449e83cce91 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:12.271712 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 184/377] 10.222.0.18 () {56 vars in 1376 bytes} [Mon Jul 25 19:08:11 2022] GET /identity/v3/role_assignments?group.id=7c8589271cc74ab88f7862a00a1915cb&scope.project.id=0e408b6ba932474fb12e507cab566abd&role.id=191e7e31dad148b7a0eb52d3afb5e50e => generated 260 bytes in 443 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:08:12.277173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c997324-3b12-41ca-8c87-79c3820c9adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:12.278113 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c997324-3b12-41ca-8c87-79c3820c9adf None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:12.278368 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c997324-3b12-41ca-8c87-79c3820c9adf None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:12.279305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c997324-3b12-41ca-8c87-79c3820c9adf None admin] PATH_INFO: `/v3/projects/9565eeec7ac7457abc5ee445dbf84c18` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:12.282048 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c997324-3b12-41ca-8c87-79c3820c9adf None admin] RBAC: Authorizing `identity:get_project(project_id=9565eeec7ac7457abc5ee445dbf84c18)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:12.283558 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c997324-3b12-41ca-8c87-79c3820c9adf None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:12.285979 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 194/378] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:08:12 2022] GET /identity/v3/projects/9565eeec7ac7457abc5ee445dbf84c18 => generated 344 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:08:12.296203 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c491d-809c-4a92-adc2-8449e83cce91 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:12.429178 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c491d-809c-4a92-adc2-8449e83cce91 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:12.479669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06188c84-59d9-4956-8b65-4460251c208e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:12.480967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06188c84-59d9-4956-8b65-4460251c208e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:12.481353 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06188c84-59d9-4956-8b65-4460251c208e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:12.481629 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06188c84-59d9-4956-8b65-4460251c208e None admin] PATH_INFO: `/v3/groups/f7486e94158a4a698230dda9997c9d9f` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:12.493642 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06188c84-59d9-4956-8b65-4460251c208e None admin] RBAC: Authorizing `identity:get_group(group_id=f7486e94158a4a698230dda9997c9d9f)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:12.495217 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06188c84-59d9-4956-8b65-4460251c208e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:12.497570 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 185/379] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Jul 25 19:08:12 2022] GET /identity/v3/groups/f7486e94158a4a698230dda9997c9d9f => generated 229 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:12.507325 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c997324-3b12-41ca-8c87-79c3820c9adf None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:12.718757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c997324-3b12-41ca-8c87-79c3820c9adf None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:12.788954 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92525ef4-d866-4be2-9b32-4e4dc2f29392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:12.790555 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92525ef4-d866-4be2-9b32-4e4dc2f29392 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:12.791344 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92525ef4-d866-4be2-9b32-4e4dc2f29392 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:12.792195 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92525ef4-d866-4be2-9b32-4e4dc2f29392 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:12.794266 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92525ef4-d866-4be2-9b32-4e4dc2f29392 None admin] RBAC: Adding query filter params (group.id=f7486e94158a4a698230dda9997c9d9f, role.id=ed7fec18968241efa76a0ee769d96da9, scope.project.id=9565eeec7ac7457abc5ee445dbf84c18) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:12.795738 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92525ef4-d866-4be2-9b32-4e4dc2f29392 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:12.798879 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92525ef4-d866-4be2-9b32-4e4dc2f29392 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:12.815770 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 195/380] 10.222.0.18 () {56 vars in 1376 bytes} [Mon Jul 25 19:08:12 2022] GET /identity/v3/role_assignments?group.id=f7486e94158a4a698230dda9997c9d9f&scope.project.id=9565eeec7ac7457abc5ee445dbf84c18&role.id=ed7fec18968241efa76a0ee769d96da9 => generated 260 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:13.502340 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8a49e-6214-4532-8509-d757283367bf None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:13.503124 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8a49e-6214-4532-8509-d757283367bf None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:13.503258 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8a49e-6214-4532-8509-d757283367bf None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:13.506112 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 186/381] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:08:13.513228 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8ef3c-0993-4cbd-8292-d08d2031dc30 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:13.514010 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8ef3c-0993-4cbd-8292-d08d2031dc30 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:13.514327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8ef3c-0993-4cbd-8292-d08d2031dc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:13.532392 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:13.561833 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b9a8ef3c-0993-4cbd-8292-d08d2031dc30 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:13.563983 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b9a8ef3c-0993-4cbd-8292-d08d2031dc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:13.703847 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130361d-b4e1-49fd-ad86-933fc1dbe3ef None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:13.704125 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130361d-b4e1-49fd-ad86-933fc1dbe3ef None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:13.704505 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130361d-b4e1-49fd-ad86-933fc1dbe3ef None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:13.706141 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 187/382] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:08:13.713461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364f8fd-d5b1-46b9-b360-16ee0455f3c4 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:13.713745 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364f8fd-d5b1-46b9-b360-16ee0455f3c4 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:13.714056 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364f8fd-d5b1-46b9-b360-16ee0455f3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:13.717655 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 196/383] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:08:13.743154 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:13.755994 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0364f8fd-d5b1-46b9-b360-16ee0455f3c4 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:13.757183 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0364f8fd-d5b1-46b9-b360-16ee0455f3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:13.777122 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8ef3c-0993-4cbd-8292-d08d2031dc30 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:13.949407 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 188/384] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:08:14.010462 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364f8fd-d5b1-46b9-b360-16ee0455f3c4 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:14.136077 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8ef3c-0993-4cbd-8292-d08d2031dc30 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:14.232434 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c54dbc-9829-4f60-9ca5-25b457c24c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:14.233623 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c54dbc-9829-4f60-9ca5-25b457c24c41 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:14.233623 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c54dbc-9829-4f60-9ca5-25b457c24c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:14.234131 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c54dbc-9829-4f60-9ca5-25b457c24c41 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:14.237012 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:14.272010 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364f8fd-d5b1-46b9-b360-16ee0455f3c4 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:14.276408 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c54dbc-9829-4f60-9ca5-25b457c24c41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:08:14.288592 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-89c54dbc-9829-4f60-9ca5-25b457c24c41 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 19:08:14.288592 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:14.288592 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:14.288592 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:14.288592 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:14.288592 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:14.288592 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:14.288592 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:14.288592 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:14.288592 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:14.288592 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:14.288592 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:14.288592 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:08:14.288592 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:08:14.288592 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:08:14.288592 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:08:14.290900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:08:14.290900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:08:14.290900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:08:14.290900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:08:14.290900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:14.290900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:14.290900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:14.290900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:14.290900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:14.290900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:14.290900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:14.290900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:14.290900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:14.290900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:14.290900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:14.290900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:14.292799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:14.292799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:14.292799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:14.292799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:14.292799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:08:14.292799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:08:14.292799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:08:14.292799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:08:14.292799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:08:14.292799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:08:14.292799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 19:08:14.292799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:14.292799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 197/385] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:08:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 514 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:08:14.300695 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c54dbc-9829-4f60-9ca5-25b457c24c41 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:14.347419 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51119f-803a-4836-a9ec-090292804ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:14.348891 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51119f-803a-4836-a9ec-090292804ece None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:14.349391 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51119f-803a-4836-a9ec-090292804ece None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:14.349876 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51119f-803a-4836-a9ec-090292804ece None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:14.363781 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:14.390745 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb51119f-803a-4836-a9ec-090292804ece None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:08:14.410851 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-bb51119f-803a-4836-a9ec-090292804ece None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 19:08:14.410851 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:14.410851 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:14.410851 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:14.410851 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:14.410851 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:14.410851 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:14.410851 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:14.410851 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:14.410851 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:14.410851 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:14.410851 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:14.410851 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:08:14.410851 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:08:14.410851 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:08:14.410851 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:08:14.412642 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:08:14.412642 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:08:14.412642 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:08:14.412642 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:08:14.412642 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:14.412642 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:14.412642 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:14.412642 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:14.412642 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:14.412642 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:14.412642 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:14.412642 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:14.412642 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:14.412642 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:14.412642 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:14.412642 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:14.414262 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:14.414262 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:14.414262 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:14.414262 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:14.414262 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:08:14.414262 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:08:14.414262 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:08:14.414262 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:08:14.414262 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:08:14.414262 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:08:14.414262 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 19:08:14.414262 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:14.423922 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 189/386] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Jul 25 19:08:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:08:14.431169 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51119f-803a-4836-a9ec-090292804ece None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:14.686994 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c54dbc-9829-4f60-9ca5-25b457c24c41 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:14.742696 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51119f-803a-4836-a9ec-090292804ece None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:14.781271 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ce2a6-4fa0-4d6b-a607-43a1e504665e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:14.781977 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ce2a6-4fa0-4d6b-a607-43a1e504665e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:14.782259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ce2a6-4fa0-4d6b-a607-43a1e504665e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:14.782532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ce2a6-4fa0-4d6b-a607-43a1e504665e None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:14.783298 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5ce2a6-4fa0-4d6b-a607-43a1e504665e None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:14.783632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5ce2a6-4fa0-4d6b-a607-43a1e504665e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:14.785165 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5ce2a6-4fa0-4d6b-a607-43a1e504665e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:14.811192 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 198/387] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Jul 25 19:08:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 511 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:14.816678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ce2a6-4fa0-4d6b-a607-43a1e504665e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:14.821790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee416eb7-1630-4502-9d0c-f1088780d87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:14.823607 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee416eb7-1630-4502-9d0c-f1088780d87f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:14.824435 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee416eb7-1630-4502-9d0c-f1088780d87f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:14.825189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee416eb7-1630-4502-9d0c-f1088780d87f None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:14.828140 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee416eb7-1630-4502-9d0c-f1088780d87f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:14.829223 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee416eb7-1630-4502-9d0c-f1088780d87f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:14.832169 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee416eb7-1630-4502-9d0c-f1088780d87f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:14.856999 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 190/388] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:08:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:14.867262 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee416eb7-1630-4502-9d0c-f1088780d87f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:14.937829 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:14.993675 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:15.023709 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ce2a6-4fa0-4d6b-a607-43a1e504665e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:15.082091 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee416eb7-1630-4502-9d0c-f1088780d87f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:15.092027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a09a93e-e8d7-4a53-a787-c7d7f2d3e040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:15.093647 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09a93e-e8d7-4a53-a787-c7d7f2d3e040 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:15.094121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09a93e-e8d7-4a53-a787-c7d7f2d3e040 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:15.094588 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09a93e-e8d7-4a53-a787-c7d7f2d3e040 None admin] PATH_INFO: `/v3/groups/7c8589271cc74ab88f7862a00a1915cb` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:15.097046 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a09a93e-e8d7-4a53-a787-c7d7f2d3e040 None admin] RBAC: Authorizing `identity:get_group(group_id=7c8589271cc74ab88f7862a00a1915cb)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:15.099753 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a09a93e-e8d7-4a53-a787-c7d7f2d3e040 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:15.108508 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 199/389] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Jul 25 19:08:14 2022] GET /identity/v3/groups/7c8589271cc74ab88f7862a00a1915cb => generated 226 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:08:15.111181 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae849da-7993-4eb1-ac4d-7d586607824d None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:15.111688 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae849da-7993-4eb1-ac4d-7d586607824d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:15.112170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae849da-7993-4eb1-ac4d-7d586607824d None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:15.120100 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 200/390] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:15 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:08:15.126072 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1304e-6c79-417a-a5b3-90be93247056 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:15.126523 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1304e-6c79-417a-a5b3-90be93247056 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:15.126983 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1304e-6c79-417a-a5b3-90be93247056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:15.152242 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3498dd-298d-4071-9ecb-f63032c3c73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:15.154236 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3498dd-298d-4071-9ecb-f63032c3c73e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:15.154846 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3498dd-298d-4071-9ecb-f63032c3c73e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:15.155307 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3498dd-298d-4071-9ecb-f63032c3c73e None admin] PATH_INFO: `/v3/groups/7c8589271cc74ab88f7862a00a1915cb` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:15.157528 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3498dd-298d-4071-9ecb-f63032c3c73e None admin] RBAC: Authorizing `identity:get_group(group_id=7c8589271cc74ab88f7862a00a1915cb)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:15.160118 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3498dd-298d-4071-9ecb-f63032c3c73e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:15.163504 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-88e1304e-6c79-417a-a5b3-90be93247056 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:15.165405 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-88e1304e-6c79-417a-a5b3-90be93247056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:15.172961 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 191/391] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Jul 25 19:08:14 2022] GET /identity/v3/groups/7c8589271cc74ab88f7862a00a1915cb => generated 226 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:15.174780 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3498dd-298d-4071-9ecb-f63032c3c73e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:15.315940 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 201/392] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:08:15.317521 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e1304e-6c79-417a-a5b3-90be93247056 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:15.393645 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3498dd-298d-4071-9ecb-f63032c3c73e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:15.432243 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:15.464307 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396743ae-2306-43dc-9abe-f5fe68685ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:15.465278 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:15.466837 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396743ae-2306-43dc-9abe-f5fe68685ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:15.469040 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396743ae-2306-43dc-9abe-f5fe68685ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:15.469562 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396743ae-2306-43dc-9abe-f5fe68685ab1 None admin] PATH_INFO: `/v3/projects/09f6174e875f4cf58a2eb5019ef11189` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:15.472616 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396743ae-2306-43dc-9abe-f5fe68685ab1 None admin] RBAC: Authorizing `identity:get_project(project_id=09f6174e875f4cf58a2eb5019ef11189)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:15.476804 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396743ae-2306-43dc-9abe-f5fe68685ab1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:15.483764 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 192/393] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:08:15 2022] GET /identity/v3/projects/09f6174e875f4cf58a2eb5019ef11189 => generated 298 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:15.484810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396743ae-2306-43dc-9abe-f5fe68685ab1 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:15.543713 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e1304e-6c79-417a-a5b3-90be93247056 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:15.634503 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0cc8ba-ba66-44d9-934c-3f24868b2f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:15.635614 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0cc8ba-ba66-44d9-934c-3f24868b2f71 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:15.635908 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0cc8ba-ba66-44d9-934c-3f24868b2f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:15.636182 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0cc8ba-ba66-44d9-934c-3f24868b2f71 None admin] PATH_INFO: `/v3/projects/09f6174e875f4cf58a2eb5019ef11189` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:15.640028 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0cc8ba-ba66-44d9-934c-3f24868b2f71 None admin] RBAC: Authorizing `identity:get_project(project_id=09f6174e875f4cf58a2eb5019ef11189)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:15.641563 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0cc8ba-ba66-44d9-934c-3f24868b2f71 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:15.647608 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 202/394] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:08:15 2022] GET /identity/v3/projects/09f6174e875f4cf58a2eb5019ef11189 => generated 298 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:15.647608 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0cc8ba-ba66-44d9-934c-3f24868b2f71 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:15.673512 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396743ae-2306-43dc-9abe-f5fe68685ab1 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:15.733278 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd7c567-b736-413e-8c88-f5c29482d04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:15.734589 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7c567-b736-413e-8c88-f5c29482d04a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:15.735061 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7c567-b736-413e-8c88-f5c29482d04a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:15.735538 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7c567-b736-413e-8c88-f5c29482d04a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:15.751126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd7c567-b736-413e-8c88-f5c29482d04a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:08:15.760555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-8bd7c567-b736-413e-8c88-f5c29482d04a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 19:08:15.760555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:15.760555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:15.760555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:15.760555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:15.760555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:15.760555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:15.760555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:15.760555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:15.760555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:15.760555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:15.760555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:15.760555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:08:15.760555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:08:15.760555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:08:15.760555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:08:15.761725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:08:15.761725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:08:15.761725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:08:15.761725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:08:15.761725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:15.761725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:15.761725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:15.761725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:15.761725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:15.761725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:15.761725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:15.761725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:15.761725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:15.761725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:15.761725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:15.761725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:15.762636 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:15.762636 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:15.762636 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:15.762636 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:15.762636 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:08:15.762636 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:08:15.762636 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:08:15.762636 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:08:15.762636 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:08:15.762636 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:08:15.762636 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 19:08:15.762636 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:15.765765 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 193/395] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:08:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:15.767562 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7cebf-625c-4cdd-ad75-1661e1cc798d None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:15.768052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7cebf-625c-4cdd-ad75-1661e1cc798d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:15.768469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7cebf-625c-4cdd-ad75-1661e1cc798d None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:15.770250 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 194/396] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:08:15.771308 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7cebf-625c-4cdd-ad75-1661e1cc798d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:15.831436 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0cc8ba-ba66-44d9-934c-3f24868b2f71 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:15.849730 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:15.880847 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:15.907742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f4dfd0-d356-4a86-9020-8c34d3025874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:15.909016 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4dfd0-d356-4a86-9020-8c34d3025874 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:15.909458 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4dfd0-d356-4a86-9020-8c34d3025874 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:15.909941 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4dfd0-d356-4a86-9020-8c34d3025874 None admin] PATH_INFO: `/v3/projects/09f6174e875f4cf58a2eb5019ef11189/groups/7c8589271cc74ab88f7862a00a1915cb/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:15.913778 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f4dfd0-d356-4a86-9020-8c34d3025874 None admin] RBAC: Authorizing `identity:create_grant(project_id=09f6174e875f4cf58a2eb5019ef11189, group_id=7c8589271cc74ab88f7862a00a1915cb, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:15.916624 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f4dfd0-d356-4a86-9020-8c34d3025874 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:15.932628 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7cebf-625c-4cdd-ad75-1661e1cc798d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:15.951696 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 203/397] 10.222.0.18 () {58 vars in 1442 bytes} [Mon Jul 25 19:08:15 2022] PUT /identity/v3/projects/09f6174e875f4cf58a2eb5019ef11189/groups/7c8589271cc74ab88f7862a00a1915cb/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:08:15.951696 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f4dfd0-d356-4a86-9020-8c34d3025874 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:16.005341 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f57da-0813-42f2-be32-4b8478584c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:16.006654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f57da-0813-42f2-be32-4b8478584c29 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:16.007011 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f57da-0813-42f2-be32-4b8478584c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:16.007967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f57da-0813-42f2-be32-4b8478584c29 None admin] PATH_INFO: `/v3/projects/09f6174e875f4cf58a2eb5019ef11189/groups/7c8589271cc74ab88f7862a00a1915cb/roles/191e7e31dad148b7a0eb52d3afb5e50e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:16.011100 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f57da-0813-42f2-be32-4b8478584c29 None admin] RBAC: Authorizing `identity:create_grant(project_id=09f6174e875f4cf58a2eb5019ef11189, group_id=7c8589271cc74ab88f7862a00a1915cb, role_id=191e7e31dad148b7a0eb52d3afb5e50e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:16.013385 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f57da-0813-42f2-be32-4b8478584c29 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:16.043529 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 195/398] 10.222.0.18 () {58 vars in 1442 bytes} [Mon Jul 25 19:08:15 2022] PUT /identity/v3/projects/09f6174e875f4cf58a2eb5019ef11189/groups/7c8589271cc74ab88f7862a00a1915cb/roles/191e7e31dad148b7a0eb52d3afb5e50e => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:08:16.046689 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca810d8-b97f-4cab-ac11-0207ff0684ed None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:16.047021 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca810d8-b97f-4cab-ac11-0207ff0684ed None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:16.047462 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca810d8-b97f-4cab-ac11-0207ff0684ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:16.081682 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2ca810d8-b97f-4cab-ac11-0207ff0684ed None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:16.082030 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2ca810d8-b97f-4cab-ac11-0207ff0684ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:16.171635 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f4dfd0-d356-4a86-9020-8c34d3025874 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:16.192545 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 196/399] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:08:16.264586 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7adb6-06a2-484c-94cd-7d63cec50198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:16.265700 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7adb6-06a2-484c-94cd-7d63cec50198 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:16.266209 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7adb6-06a2-484c-94cd-7d63cec50198 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:16.266730 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7adb6-06a2-484c-94cd-7d63cec50198 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:16.268539 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e7adb6-06a2-484c-94cd-7d63cec50198 None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:16.269174 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e7adb6-06a2-484c-94cd-7d63cec50198 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:16.270047 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca810d8-b97f-4cab-ac11-0207ff0684ed None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:16.272204 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e7adb6-06a2-484c-94cd-7d63cec50198 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:16.288916 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 204/400] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Jul 25 19:08:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:16.290479 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c99c7e-2ec0-4ec6-9efe-e11237d266a6 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:16.290772 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c99c7e-2ec0-4ec6-9efe-e11237d266a6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:16.291054 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c99c7e-2ec0-4ec6-9efe-e11237d266a6 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:16.292793 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 205/401] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:08:16.298951 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c99c7e-2ec0-4ec6-9efe-e11237d266a6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:16.379105 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:16.383988 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:16.501394 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca810d8-b97f-4cab-ac11-0207ff0684ed None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:16.523669 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c99c7e-2ec0-4ec6-9efe-e11237d266a6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:16.561657 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7fcbe6-934d-4f81-98a0-c8d6d69aa7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:16.562459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fcbe6-934d-4f81-98a0-c8d6d69aa7de None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:16.562745 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fcbe6-934d-4f81-98a0-c8d6d69aa7de None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:16.563023 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fcbe6-934d-4f81-98a0-c8d6d69aa7de None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:16.576756 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7fcbe6-934d-4f81-98a0-c8d6d69aa7de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:08:16.586161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-5a7fcbe6-934d-4f81-98a0-c8d6d69aa7de None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 19:08:16.586161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:16.586161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:16.586161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:16.586161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:16.586161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:16.586161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:16.586161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:16.586161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:16.586161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:16.586161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:16.586161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:16.586161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:08:16.586161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:08:16.586161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:08:16.586161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:08:16.588222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:08:16.588222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:08:16.588222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:08:16.588222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:08:16.588222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:16.588222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:16.588222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:16.588222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:16.588222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:16.588222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:16.588222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:16.588222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:16.588222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:16.588222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:16.588222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:16.588222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:16.590005 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:16.590005 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:16.590005 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:16.590005 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:16.590005 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:08:16.590005 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:08:16.590005 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:08:16.590005 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:08:16.590005 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:08:16.590005 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:08:16.590005 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 19:08:16.590005 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:16.593721 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 197/402] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Jul 25 19:08:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:16.595093 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4293023-e1ae-4e63-9182-5d5216a753f6 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:16.595370 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4293023-e1ae-4e63-9182-5d5216a753f6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:16.595690 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4293023-e1ae-4e63-9182-5d5216a753f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:16.615598 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9581e00-8410-46bf-a9b3-183d88d6ecf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:16.617568 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9581e00-8410-46bf-a9b3-183d88d6ecf3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:16.618319 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9581e00-8410-46bf-a9b3-183d88d6ecf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:16.619057 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9581e00-8410-46bf-a9b3-183d88d6ecf3 None admin] PATH_INFO: `/v3/groups/7c8589271cc74ab88f7862a00a1915cb` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:16.621703 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9581e00-8410-46bf-a9b3-183d88d6ecf3 None admin] RBAC: Authorizing `identity:get_group(group_id=7c8589271cc74ab88f7862a00a1915cb)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:16.624542 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c4293023-e1ae-4e63-9182-5d5216a753f6 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:16.625076 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9581e00-8410-46bf-a9b3-183d88d6ecf3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:16.626808 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c4293023-e1ae-4e63-9182-5d5216a753f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:16.629205 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 206/403] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Jul 25 19:08:16 2022] GET /identity/v3/groups/7c8589271cc74ab88f7862a00a1915cb => generated 226 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:16.631142 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9581e00-8410-46bf-a9b3-183d88d6ecf3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:16.771555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 198/404] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:08:16.774533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4293023-e1ae-4e63-9182-5d5216a753f6 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:16.813252 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9581e00-8410-46bf-a9b3-183d88d6ecf3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:16.831306 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:16.864614 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:16.867175 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93235762-4e72-48ba-8b80-9b4fb773e366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:16.868747 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93235762-4e72-48ba-8b80-9b4fb773e366 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:16.869221 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93235762-4e72-48ba-8b80-9b4fb773e366 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:16.869841 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93235762-4e72-48ba-8b80-9b4fb773e366 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:16.870996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93235762-4e72-48ba-8b80-9b4fb773e366 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:16.871606 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93235762-4e72-48ba-8b80-9b4fb773e366 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:16.874982 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93235762-4e72-48ba-8b80-9b4fb773e366 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:16.893690 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 207/405] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:08:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:16.894875 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93235762-4e72-48ba-8b80-9b4fb773e366 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:16.997070 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4293023-e1ae-4e63-9182-5d5216a753f6 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:17.043999 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e25ae-2655-4962-94a0-cdf0bf8389a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:17.045931 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e25ae-2655-4962-94a0-cdf0bf8389a2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:17.046483 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e25ae-2655-4962-94a0-cdf0bf8389a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:17.046964 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e25ae-2655-4962-94a0-cdf0bf8389a2 None admin] PATH_INFO: `/v3/projects/0e408b6ba932474fb12e507cab566abd` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:17.048817 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3e25ae-2655-4962-94a0-cdf0bf8389a2 None admin] RBAC: Authorizing `identity:get_project(project_id=0e408b6ba932474fb12e507cab566abd)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:17.050788 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3e25ae-2655-4962-94a0-cdf0bf8389a2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:17.053410 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 199/406] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:08:16 2022] GET /identity/v3/projects/0e408b6ba932474fb12e507cab566abd => generated 302 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:17.054458 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e25ae-2655-4962-94a0-cdf0bf8389a2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:17.094642 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93235762-4e72-48ba-8b80-9b4fb773e366 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:17.146594 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73091bcf-5048-42ad-82e9-4b7cb5be2256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:17.148223 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73091bcf-5048-42ad-82e9-4b7cb5be2256 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:17.148783 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73091bcf-5048-42ad-82e9-4b7cb5be2256 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:17.149332 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73091bcf-5048-42ad-82e9-4b7cb5be2256 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:17.171472 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73091bcf-5048-42ad-82e9-4b7cb5be2256 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:08:17.181654 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-73091bcf-5048-42ad-82e9-4b7cb5be2256 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:08:17.181654 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:17.181654 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:17.181654 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:17.181654 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:17.181654 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:17.181654 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:17.181654 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:17.181654 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:17.181654 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:17.181654 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:17.181654 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:17.181654 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:08:17.181654 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:08:17.181654 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:08:17.181654 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:08:17.183038 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:08:17.183038 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:08:17.183038 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:08:17.183038 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:08:17.183038 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:17.183038 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:17.183038 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:17.183038 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:17.183038 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:17.183038 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:17.183038 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:17.183038 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:17.183038 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:17.183038 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:17.183038 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:17.183038 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:17.183945 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:17.183945 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:17.183945 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:17.183945 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:17.183945 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:08:17.183945 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:08:17.183945 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:08:17.183945 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:08:17.183945 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:08:17.183945 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:08:17.183945 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:08:17.183945 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:17.187258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 208/407] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Jul 25 19:08:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:17.188684 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73091bcf-5048-42ad-82e9-4b7cb5be2256 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:17.269805 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:17.277013 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e25ae-2655-4962-94a0-cdf0bf8389a2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:17.280438 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:17.327570 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35c777-4e33-49fb-bfc4-968a0332e2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:17.328942 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35c777-4e33-49fb-bfc4-968a0332e2cf None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:17.329445 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35c777-4e33-49fb-bfc4-968a0332e2cf None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:17.329931 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35c777-4e33-49fb-bfc4-968a0332e2cf None admin] PATH_INFO: `/v3/groups/7c8589271cc74ab88f7862a00a1915cb` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:17.331576 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35c777-4e33-49fb-bfc4-968a0332e2cf None admin] RBAC: Authorizing `identity:get_group(group_id=7c8589271cc74ab88f7862a00a1915cb)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:17.333499 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35c777-4e33-49fb-bfc4-968a0332e2cf None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:17.340487 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 200/408] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Jul 25 19:08:17 2022] GET /identity/v3/groups/7c8589271cc74ab88f7862a00a1915cb => generated 226 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:17.342888 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35c777-4e33-49fb-bfc4-968a0332e2cf None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:17.397317 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73091bcf-5048-42ad-82e9-4b7cb5be2256 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:17.453323 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b318c-f1ae-4125-8708-c5a1b4260551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:17.454407 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b318c-f1ae-4125-8708-c5a1b4260551 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:17.454699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b318c-f1ae-4125-8708-c5a1b4260551 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:17.454982 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b318c-f1ae-4125-8708-c5a1b4260551 None admin] PATH_INFO: `/v3/projects/0e408b6ba932474fb12e507cab566abd/groups/7c8589271cc74ab88f7862a00a1915cb/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:17.457329 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87b318c-f1ae-4125-8708-c5a1b4260551 None admin] RBAC: Authorizing `identity:create_grant(project_id=0e408b6ba932474fb12e507cab566abd, group_id=7c8589271cc74ab88f7862a00a1915cb, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:17.459367 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87b318c-f1ae-4125-8708-c5a1b4260551 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:17.477523 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35c777-4e33-49fb-bfc4-968a0332e2cf None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:17.479333 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 209/409] 10.222.0.18 () {58 vars in 1442 bytes} [Mon Jul 25 19:08:17 2022] PUT /identity/v3/projects/0e408b6ba932474fb12e507cab566abd/groups/7c8589271cc74ab88f7862a00a1915cb/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:08:17.484592 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b318c-f1ae-4125-8708-c5a1b4260551 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:17.571716 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80039bf4-628b-4e34-bffa-fe0a8404cbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:17.572882 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80039bf4-628b-4e34-bffa-fe0a8404cbed None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:17.573654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80039bf4-628b-4e34-bffa-fe0a8404cbed None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:17.577069 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80039bf4-628b-4e34-bffa-fe0a8404cbed None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:17.578311 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80039bf4-628b-4e34-bffa-fe0a8404cbed None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:17.578927 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80039bf4-628b-4e34-bffa-fe0a8404cbed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:17.582450 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80039bf4-628b-4e34-bffa-fe0a8404cbed None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:17.604041 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 201/410] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:08:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:17.612200 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80039bf4-628b-4e34-bffa-fe0a8404cbed None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:17.691056 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:17.733501 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:17.746549 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b318c-f1ae-4125-8708-c5a1b4260551 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:17.793800 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbff30f8-fd44-4643-aca8-1fa5634bb1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:17.795737 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff30f8-fd44-4643-aca8-1fa5634bb1d4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:17.796474 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff30f8-fd44-4643-aca8-1fa5634bb1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:17.796980 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff30f8-fd44-4643-aca8-1fa5634bb1d4 None admin] PATH_INFO: `/v3/projects/0e408b6ba932474fb12e507cab566abd` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:17.798610 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbff30f8-fd44-4643-aca8-1fa5634bb1d4 None admin] RBAC: Authorizing `identity:get_project(project_id=0e408b6ba932474fb12e507cab566abd)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:17.800662 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbff30f8-fd44-4643-aca8-1fa5634bb1d4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:17.803096 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 210/411] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:08:17 2022] GET /identity/v3/projects/0e408b6ba932474fb12e507cab566abd => generated 302 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:17.812741 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbff30f8-fd44-4643-aca8-1fa5634bb1d4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:17.857033 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80039bf4-628b-4e34-bffa-fe0a8404cbed None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:17.974377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbff30f8-fd44-4643-aca8-1fa5634bb1d4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:17.980783 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0276d19a-27ab-4488-a5e9-e72abb198972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:17.982941 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276d19a-27ab-4488-a5e9-e72abb198972 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:17.983446 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276d19a-27ab-4488-a5e9-e72abb198972 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:17.984002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276d19a-27ab-4488-a5e9-e72abb198972 None admin] PATH_INFO: `/v3/groups/f7486e94158a4a698230dda9997c9d9f` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:17.993303 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0276d19a-27ab-4488-a5e9-e72abb198972 None admin] RBAC: Authorizing `identity:get_group(group_id=f7486e94158a4a698230dda9997c9d9f)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:17.995811 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0276d19a-27ab-4488-a5e9-e72abb198972 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:18.008145 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 202/412] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Jul 25 19:08:17 2022] GET /identity/v3/groups/f7486e94158a4a698230dda9997c9d9f => generated 229 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:18.017761 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0276d19a-27ab-4488-a5e9-e72abb198972 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:18.027602 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07937b5-f632-40bb-8b0b-a8f50c5a3d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:18.029662 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07937b5-f632-40bb-8b0b-a8f50c5a3d1b None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:18.031394 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07937b5-f632-40bb-8b0b-a8f50c5a3d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:18.032232 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07937b5-f632-40bb-8b0b-a8f50c5a3d1b None admin] PATH_INFO: `/v3/projects/0e408b6ba932474fb12e507cab566abd/groups/7c8589271cc74ab88f7862a00a1915cb/roles/191e7e31dad148b7a0eb52d3afb5e50e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:18.035047 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07937b5-f632-40bb-8b0b-a8f50c5a3d1b None admin] RBAC: Authorizing `identity:create_grant(project_id=0e408b6ba932474fb12e507cab566abd, group_id=7c8589271cc74ab88f7862a00a1915cb, role_id=191e7e31dad148b7a0eb52d3afb5e50e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:18.037129 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07937b5-f632-40bb-8b0b-a8f50c5a3d1b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:18.060975 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 211/413] 10.222.0.18 () {58 vars in 1442 bytes} [Mon Jul 25 19:08:17 2022] PUT /identity/v3/projects/0e408b6ba932474fb12e507cab566abd/groups/7c8589271cc74ab88f7862a00a1915cb/roles/191e7e31dad148b7a0eb52d3afb5e50e => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:08:18.256042 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:18.269958 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0276d19a-27ab-4488-a5e9-e72abb198972 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:18.326097 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d371d7da-a160-429a-b66d-4615a012a615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:18.326988 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d371d7da-a160-429a-b66d-4615a012a615 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:18.327272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d371d7da-a160-429a-b66d-4615a012a615 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:18.327573 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d371d7da-a160-429a-b66d-4615a012a615 None admin] PATH_INFO: `/v3/projects/9565eeec7ac7457abc5ee445dbf84c18` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:18.328980 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d371d7da-a160-429a-b66d-4615a012a615 None admin] RBAC: Authorizing `identity:get_project(project_id=9565eeec7ac7457abc5ee445dbf84c18)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:18.330476 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d371d7da-a160-429a-b66d-4615a012a615 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:18.332817 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 203/414] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:08:18 2022] GET /identity/v3/projects/9565eeec7ac7457abc5ee445dbf84c18 => generated 344 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:08:18.339498 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07937b5-f632-40bb-8b0b-a8f50c5a3d1b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:18.378108 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:18.489657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07937b5-f632-40bb-8b0b-a8f50c5a3d1b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:18.545903 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a742bb0-cfa0-4697-981f-7a61328054c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:18.546960 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a742bb0-cfa0-4697-981f-7a61328054c7 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:18.547255 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a742bb0-cfa0-4697-981f-7a61328054c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:18.547551 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a742bb0-cfa0-4697-981f-7a61328054c7 None admin] PATH_INFO: `/v3/projects/9565eeec7ac7457abc5ee445dbf84c18/groups/f7486e94158a4a698230dda9997c9d9f/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:18.550533 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a742bb0-cfa0-4697-981f-7a61328054c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=9565eeec7ac7457abc5ee445dbf84c18, group_id=f7486e94158a4a698230dda9997c9d9f, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:18.552409 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a742bb0-cfa0-4697-981f-7a61328054c7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:18.575079 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 212/415] 10.222.0.18 () {58 vars in 1442 bytes} [Mon Jul 25 19:08:18 2022] PUT /identity/v3/projects/9565eeec7ac7457abc5ee445dbf84c18/groups/f7486e94158a4a698230dda9997c9d9f/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:08:18.666741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c46e6e-c6eb-42ee-b2fc-8fd9758bf32f None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:18.667203 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c46e6e-c6eb-42ee-b2fc-8fd9758bf32f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:18.667731 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c46e6e-c6eb-42ee-b2fc-8fd9758bf32f None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:18.670863 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 204/416] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:08:18.677794 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a5362-ef5a-4942-8261-7151563f6021 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:18.678282 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a5362-ef5a-4942-8261-7151563f6021 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:18.678446 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a5362-ef5a-4942-8261-7151563f6021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:18.701137 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-338a5362-ef5a-4942-8261-7151563f6021 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:18.702574 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-338a5362-ef5a-4942-8261-7151563f6021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:18.856968 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 213/417] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 19:08:18.899160 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c46e6e-c6eb-42ee-b2fc-8fd9758bf32f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:19.113834 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c46e6e-c6eb-42ee-b2fc-8fd9758bf32f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:19.179599 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f4ab87-3ec5-423b-8768-0be2de88cb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:19.180951 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4ab87-3ec5-423b-8768-0be2de88cb64 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:19.181496 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4ab87-3ec5-423b-8768-0be2de88cb64 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:19.181984 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4ab87-3ec5-423b-8768-0be2de88cb64 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:19.201613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f4ab87-3ec5-423b-8768-0be2de88cb64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:08:19.215499 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-f8f4ab87-3ec5-423b-8768-0be2de88cb64 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 19:08:19.215499 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:19.215499 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:19.215499 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:19.215499 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:19.215499 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:19.215499 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:19.215499 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:19.215499 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:19.215499 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:19.215499 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:19.215499 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:19.215499 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:08:19.215499 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:08:19.215499 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:08:19.215499 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:08:19.217364 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:08:19.217364 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:08:19.217364 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:08:19.217364 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:08:19.217364 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:19.217364 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:19.217364 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:19.217364 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:19.217364 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:19.217364 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:19.217364 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:19.217364 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:19.217364 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:19.217364 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:19.217364 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:19.217364 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:19.218180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:19.218180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:19.218180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:19.218180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:19.218180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:08:19.218180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:08:19.218180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:08:19.218180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:08:19.218180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:08:19.218180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:08:19.218180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 19:08:19.218180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:19.220944 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 205/418] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Jul 25 19:08:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:19.231312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a5362-ef5a-4942-8261-7151563f6021 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:19.283449 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:19.465513 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a5362-ef5a-4942-8261-7151563f6021 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:19.546425 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cdf1d1-6ccc-4963-ab2d-a4051babff38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:19.547209 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdf1d1-6ccc-4963-ab2d-a4051babff38 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:19.547647 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdf1d1-6ccc-4963-ab2d-a4051babff38 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:19.547973 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdf1d1-6ccc-4963-ab2d-a4051babff38 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:19.549009 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cdf1d1-6ccc-4963-ab2d-a4051babff38 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:19.549326 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cdf1d1-6ccc-4963-ab2d-a4051babff38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:19.551010 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cdf1d1-6ccc-4963-ab2d-a4051babff38 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:19.572180 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 214/419] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:08:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:19.584310 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f4ab87-3ec5-423b-8768-0be2de88cb64 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:19.656884 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:19.713223 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf4570-20b5-42a1-bbe9-8c054033d219 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:19.713685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf4570-20b5-42a1-bbe9-8c054033d219 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:19.713861 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf4570-20b5-42a1-bbe9-8c054033d219 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:19.716773 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 215/420] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:08:19.723557 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c7c8f-6cef-44ba-8969-28cfd69f1a95 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:19.723959 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c7c8f-6cef-44ba-8969-28cfd69f1a95 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:19.724222 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c7c8f-6cef-44ba-8969-28cfd69f1a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:19.769772 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ea1c7c8f-6cef-44ba-8969-28cfd69f1a95 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:19.773138 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ea1c7c8f-6cef-44ba-8969-28cfd69f1a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:19.791477 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f4ab87-3ec5-423b-8768-0be2de88cb64 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:19.839209 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27af27ec-2f32-4cf9-990e-5719d1608104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:19.839955 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af27ec-2f32-4cf9-990e-5719d1608104 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:19.840288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af27ec-2f32-4cf9-990e-5719d1608104 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:19.840840 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af27ec-2f32-4cf9-990e-5719d1608104 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:19.929092 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27af27ec-2f32-4cf9-990e-5719d1608104 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:19.930822 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27af27ec-2f32-4cf9-990e-5719d1608104 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:19.968142 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:19.974532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 216/421] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:08:20.023310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c7c8f-6cef-44ba-8969-28cfd69f1a95 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:20.031251 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:20.118917 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 206/422] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:08:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 19:08:20.129079 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27af27ec-2f32-4cf9-990e-5719d1608104 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:20.279727 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c7c8f-6cef-44ba-8969-28cfd69f1a95 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:20.289805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27af27ec-2f32-4cf9-990e-5719d1608104 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:20.363976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:20.367568 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a9039-e83d-4f0a-ac1d-520a405618af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:20.369395 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a9039-e83d-4f0a-ac1d-520a405618af None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:20.370588 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a9039-e83d-4f0a-ac1d-520a405618af None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:20.374590 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a9039-e83d-4f0a-ac1d-520a405618af None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:20.398171 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28781719-6382-4795-876a-11b1c869b5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:20.399070 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28781719-6382-4795-876a-11b1c869b5b4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:20.399386 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28781719-6382-4795-876a-11b1c869b5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:20.399701 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28781719-6382-4795-876a-11b1c869b5b4 None admin] PATH_INFO: `/v3/projects/09f6174e875f4cf58a2eb5019ef11189` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:20.403546 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a9039-e83d-4f0a-ac1d-520a405618af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:08:20.404459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28781719-6382-4795-876a-11b1c869b5b4 None admin] RBAC: Authorizing `identity:get_project(project_id=09f6174e875f4cf58a2eb5019ef11189)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:20.405984 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28781719-6382-4795-876a-11b1c869b5b4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:20.408400 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 207/423] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:08:20 2022] GET /identity/v3/projects/09f6174e875f4cf58a2eb5019ef11189 => generated 298 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:08:20.414196 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28781719-6382-4795-876a-11b1c869b5b4 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:20.420841 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-768a9039-e83d-4f0a-ac1d-520a405618af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 19:08:20.420841 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:20.420841 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:20.420841 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:20.420841 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:20.420841 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:20.420841 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:20.420841 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:20.420841 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:20.420841 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:20.420841 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:20.420841 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:20.420841 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:08:20.420841 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:08:20.420841 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:08:20.420841 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:08:20.422684 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:08:20.422684 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:08:20.422684 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:08:20.422684 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:08:20.422684 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:20.422684 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:20.422684 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:20.422684 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:20.422684 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:20.422684 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:20.422684 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:20.422684 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:20.422684 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:20.422684 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:20.422684 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:20.422684 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:20.424327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:20.424327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:20.424327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:20.424327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:20.424327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:08:20.424327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:08:20.424327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:08:20.424327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:08:20.424327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:08:20.424327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:08:20.424327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 19:08:20.424327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:20.429323 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 217/424] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:08:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 407 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:08:20.440706 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a9039-e83d-4f0a-ac1d-520a405618af None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:20.603759 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:20.704432 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a9039-e83d-4f0a-ac1d-520a405618af None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:20.730079 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28781719-6382-4795-876a-11b1c869b5b4 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:20.783710 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf2e068-4dfb-4fc9-81b3-7413b1f9137e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:20.784889 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf2e068-4dfb-4fc9-81b3-7413b1f9137e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:20.785402 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf2e068-4dfb-4fc9-81b3-7413b1f9137e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:20.785836 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf2e068-4dfb-4fc9-81b3-7413b1f9137e None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:20.787200 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf2e068-4dfb-4fc9-81b3-7413b1f9137e None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:20.787767 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf2e068-4dfb-4fc9-81b3-7413b1f9137e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:20.790344 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf2e068-4dfb-4fc9-81b3-7413b1f9137e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:20.822408 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 218/425] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Jul 25 19:08:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:20.833149 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf2e068-4dfb-4fc9-81b3-7413b1f9137e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:20.859396 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4628b93e-ed80-46bc-a753-14e63f5d16f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:20.861327 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4628b93e-ed80-46bc-a753-14e63f5d16f1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:20.862132 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4628b93e-ed80-46bc-a753-14e63f5d16f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:20.862847 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4628b93e-ed80-46bc-a753-14e63f5d16f1 None admin] PATH_INFO: `/v3/groups/7c8589271cc74ab88f7862a00a1915cb` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:20.865567 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4628b93e-ed80-46bc-a753-14e63f5d16f1 None admin] RBAC: Authorizing `identity:get_group(group_id=7c8589271cc74ab88f7862a00a1915cb)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:20.868812 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4628b93e-ed80-46bc-a753-14e63f5d16f1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:20.881483 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 208/426] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Jul 25 19:08:20 2022] GET /identity/v3/groups/7c8589271cc74ab88f7862a00a1915cb => generated 226 bytes in 468 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:20.888420 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4628b93e-ed80-46bc-a753-14e63f5d16f1 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:21.028570 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:21.107420 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf2e068-4dfb-4fc9-81b3-7413b1f9137e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:21.114331 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4628b93e-ed80-46bc-a753-14e63f5d16f1 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:21.140614 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:21.157090 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a268aef1-7754-43f5-81f7-74a1e21143d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:21.158516 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268aef1-7754-43f5-81f7-74a1e21143d2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:21.159219 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268aef1-7754-43f5-81f7-74a1e21143d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:21.159936 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268aef1-7754-43f5-81f7-74a1e21143d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:21.194276 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddfe18e-be18-41a3-afdc-15cae14dbbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:21.196063 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddfe18e-be18-41a3-afdc-15cae14dbbdf None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:21.196916 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddfe18e-be18-41a3-afdc-15cae14dbbdf None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:21.197697 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddfe18e-be18-41a3-afdc-15cae14dbbdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:21.199922 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddfe18e-be18-41a3-afdc-15cae14dbbdf None admin] RBAC: Adding query filter params (group.id=7c8589271cc74ab88f7862a00a1915cb, role.id=191e7e31dad148b7a0eb52d3afb5e50e, scope.project.id=09f6174e875f4cf58a2eb5019ef11189) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:21.200912 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddfe18e-be18-41a3-afdc-15cae14dbbdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:21.204369 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddfe18e-be18-41a3-afdc-15cae14dbbdf None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:21.218639 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 209/427] 10.222.0.18 () {56 vars in 1376 bytes} [Mon Jul 25 19:08:20 2022] GET /identity/v3/role_assignments?group.id=7c8589271cc74ab88f7862a00a1915cb&scope.project.id=09f6174e875f4cf58a2eb5019ef11189&role.id=191e7e31dad148b7a0eb52d3afb5e50e => generated 611 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:08:21.221646 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290bdac6-7883-4e9e-b958-e2a6f0bd6b82 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:21.222587 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290bdac6-7883-4e9e-b958-e2a6f0bd6b82 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:21.223377 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290bdac6-7883-4e9e-b958-e2a6f0bd6b82 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:21.226416 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 210/428] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:08:21.236700 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4227ec-a378-4ee3-99be-e3f86e5a12e4 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:21.237272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4227ec-a378-4ee3-99be-e3f86e5a12e4 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:21.237703 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4227ec-a378-4ee3-99be-e3f86e5a12e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:21.274627 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a268aef1-7754-43f5-81f7-74a1e21143d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:21.276490 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a268aef1-7754-43f5-81f7-74a1e21143d2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:21.287764 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5f4227ec-a378-4ee3-99be-e3f86e5a12e4 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:21.289143 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5f4227ec-a378-4ee3-99be-e3f86e5a12e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:21.417757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 219/429] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:08:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 19:08:21.420959 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab81a26-8a4f-46e0-8c78-06b40f0031e3 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:21.421500 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab81a26-8a4f-46e0-8c78-06b40f0031e3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:21.421941 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab81a26-8a4f-46e0-8c78-06b40f0031e3 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:21.424011 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 220/430] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:08:21.433969 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab81a26-8a4f-46e0-8c78-06b40f0031e3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:21.464278 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 211/431] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:08:21.467994 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a9e30-107b-456a-ae28-658f467e25aa None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:21.468595 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a9e30-107b-456a-ae28-658f467e25aa None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:21.469167 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a9e30-107b-456a-ae28-658f467e25aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:21.480507 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:21.510109 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-fe6a9e30-107b-456a-ae28-658f467e25aa None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:21.511561 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fe6a9e30-107b-456a-ae28-658f467e25aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:21.601875 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:21.620293 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab81a26-8a4f-46e0-8c78-06b40f0031e3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:21.625247 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 212/432] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:08:21.627151 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a9e30-107b-456a-ae28-658f467e25aa None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:21.688588 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e44d8-a654-4e95-a326-28254fea2df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:21.689814 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e44d8-a654-4e95-a326-28254fea2df2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:21.690278 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e44d8-a654-4e95-a326-28254fea2df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:21.690561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e44d8-a654-4e95-a326-28254fea2df2 None admin] PATH_INFO: `/v3/projects/09f6174e875f4cf58a2eb5019ef11189` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:21.692045 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267e44d8-a654-4e95-a326-28254fea2df2 None admin] RBAC: Authorizing `identity:get_project(project_id=09f6174e875f4cf58a2eb5019ef11189)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:21.693566 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267e44d8-a654-4e95-a326-28254fea2df2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:21.695652 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 221/433] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:08:21 2022] GET /identity/v3/projects/09f6174e875f4cf58a2eb5019ef11189 => generated 298 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:21.696877 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e44d8-a654-4e95-a326-28254fea2df2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:21.815967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a9e30-107b-456a-ae28-658f467e25aa None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:21.870998 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:21.883302 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e44d8-a654-4e95-a326-28254fea2df2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:21.891165 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da63b2a5-7b27-4cfe-aa8c-4275d8c921f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:21.892634 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63b2a5-7b27-4cfe-aa8c-4275d8c921f3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:21.893432 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63b2a5-7b27-4cfe-aa8c-4275d8c921f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:21.894170 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63b2a5-7b27-4cfe-aa8c-4275d8c921f3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:21.915243 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da63b2a5-7b27-4cfe-aa8c-4275d8c921f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:08:21.931578 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-da63b2a5-7b27-4cfe-aa8c-4275d8c921f3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 19:08:21.931578 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:21.931578 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:21.931578 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:21.931578 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:21.931578 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:21.931578 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:21.931578 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:21.931578 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:21.931578 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:21.931578 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:21.931578 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:21.931578 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:08:21.931578 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:08:21.931578 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:08:21.931578 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:08:21.933469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:08:21.933469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:08:21.933469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:08:21.933469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:08:21.933469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:21.933469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:21.933469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:21.933469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:21.933469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:21.933469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:21.933469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:21.933469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:21.933469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:21.933469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:21.933469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:21.933469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:21.935713 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:21.935713 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:21.935713 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:21.935713 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:21.935713 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:08:21.935713 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:08:21.935713 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:08:21.935713 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:08:21.935713 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:08:21.935713 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:08:21.935713 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 19:08:21.935713 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:21.939416 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d57797b-3fc3-4bf3-83e1-ab432a8ad6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:21.939416 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57797b-3fc3-4bf3-83e1-ab432a8ad6d6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:21.939416 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57797b-3fc3-4bf3-83e1-ab432a8ad6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:21.939416 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57797b-3fc3-4bf3-83e1-ab432a8ad6d6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:21.940400 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 213/434] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:08:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:21.940400 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da63b2a5-7b27-4cfe-aa8c-4275d8c921f3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:21.964920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d57797b-3fc3-4bf3-83e1-ab432a8ad6d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:08:21.983542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-6d57797b-3fc3-4bf3-83e1-ab432a8ad6d6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 19:08:21.983542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:21.983542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:21.983542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:21.983542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:21.983542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:21.983542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:21.983542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:21.983542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:21.983542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:21.983542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:21.983542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:21.983542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:08:21.983542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:08:21.983542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:08:21.983542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:08:21.984773 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:08:21.984773 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:08:21.984773 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:08:21.984773 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:08:21.984773 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:21.984773 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:21.984773 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:21.984773 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:21.984773 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:21.984773 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:21.984773 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:21.984773 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:21.984773 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:21.984773 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:21.984773 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:21.984773 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:21.985755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:21.985755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:21.985755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:21.985755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:21.985755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:08:21.985755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:08:21.985755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:08:21.985755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:08:21.985755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:08:21.985755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:08:21.985755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 19:08:21.985755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:21.986377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 222/435] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Jul 25 19:08:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:21.987378 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d57797b-3fc3-4bf3-83e1-ab432a8ad6d6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:22.031343 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:22.145346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da63b2a5-7b27-4cfe-aa8c-4275d8c921f3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:22.158455 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d57797b-3fc3-4bf3-83e1-ab432a8ad6d6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:22.211156 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3dd267-8388-41d2-b71a-2e8b8870c263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:22.212034 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3dd267-8388-41d2-b71a-2e8b8870c263 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:22.212455 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3dd267-8388-41d2-b71a-2e8b8870c263 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:22.212848 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30c4fc-8e2a-47c2-b3bc-7d17ed08572e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:22.213026 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3dd267-8388-41d2-b71a-2e8b8870c263 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:22.213896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3dd267-8388-41d2-b71a-2e8b8870c263 None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:22.214344 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30c4fc-8e2a-47c2-b3bc-7d17ed08572e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:22.214497 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3dd267-8388-41d2-b71a-2e8b8870c263 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:22.214848 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30c4fc-8e2a-47c2-b3bc-7d17ed08572e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:22.215338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30c4fc-8e2a-47c2-b3bc-7d17ed08572e None admin] PATH_INFO: `/v3/groups/7c8589271cc74ab88f7862a00a1915cb` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:22.217490 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3dd267-8388-41d2-b71a-2e8b8870c263 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:22.218569 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30c4fc-8e2a-47c2-b3bc-7d17ed08572e None admin] RBAC: Authorizing `identity:get_group(group_id=7c8589271cc74ab88f7862a00a1915cb)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:22.221501 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30c4fc-8e2a-47c2-b3bc-7d17ed08572e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:22.225501 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 214/436] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Jul 25 19:08:21 2022] GET /identity/v3/groups/7c8589271cc74ab88f7862a00a1915cb => generated 226 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:22.226671 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30c4fc-8e2a-47c2-b3bc-7d17ed08572e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:22.235415 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 223/437] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Jul 25 19:08:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:22.237471 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3dd267-8388-41d2-b71a-2e8b8870c263 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:22.271087 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:22.396151 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30c4fc-8e2a-47c2-b3bc-7d17ed08572e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:22.421662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:22.432504 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3dd267-8388-41d2-b71a-2e8b8870c263 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:22.447638 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2ba03e-6ed5-460b-a79d-9c4841335558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:22.448586 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ba03e-6ed5-460b-a79d-9c4841335558 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:22.448942 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ba03e-6ed5-460b-a79d-9c4841335558 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:22.449317 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ba03e-6ed5-460b-a79d-9c4841335558 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:22.450961 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2ba03e-6ed5-460b-a79d-9c4841335558 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:22.451381 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2ba03e-6ed5-460b-a79d-9c4841335558 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:22.453007 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2ba03e-6ed5-460b-a79d-9c4841335558 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:22.465015 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 215/438] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:08:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:22.466881 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2ba03e-6ed5-460b-a79d-9c4841335558 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:22.482168 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a57680-15b5-48fe-b239-6692fef606be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:22.482999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a57680-15b5-48fe-b239-6692fef606be None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:22.483401 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a57680-15b5-48fe-b239-6692fef606be None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:22.483803 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a57680-15b5-48fe-b239-6692fef606be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:22.484820 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a57680-15b5-48fe-b239-6692fef606be None admin] RBAC: Adding query filter params (group.id=7c8589271cc74ab88f7862a00a1915cb, role.id=c241bd9634e24bcbb1854c7621d19f81, scope.project.id=09f6174e875f4cf58a2eb5019ef11189) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:22.485278 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a57680-15b5-48fe-b239-6692fef606be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:22.486965 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a57680-15b5-48fe-b239-6692fef606be None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:22.494698 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 224/439] 10.222.0.18 () {56 vars in 1376 bytes} [Mon Jul 25 19:08:22 2022] GET /identity/v3/role_assignments?group.id=7c8589271cc74ab88f7862a00a1915cb&scope.project.id=09f6174e875f4cf58a2eb5019ef11189&role.id=c241bd9634e24bcbb1854c7621d19f81 => generated 611 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:22.496527 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59405de0-d451-4e31-8312-20e952ccb5ba None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:22.496976 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59405de0-d451-4e31-8312-20e952ccb5ba None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:22.497451 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59405de0-d451-4e31-8312-20e952ccb5ba None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:22.499343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 225/440] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:08:22.500248 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59405de0-d451-4e31-8312-20e952ccb5ba None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:22.652320 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2ba03e-6ed5-460b-a79d-9c4841335558 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:22.665486 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:22.698802 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760e250-140c-4b90-96bd-f4da1479437e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:22.699581 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760e250-140c-4b90-96bd-f4da1479437e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:22.699889 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760e250-140c-4b90-96bd-f4da1479437e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:22.700716 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760e250-140c-4b90-96bd-f4da1479437e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:22.703602 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59405de0-d451-4e31-8312-20e952ccb5ba None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:22.764461 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423615e-37c8-45bd-9c3f-8d28173b929c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:22.765821 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423615e-37c8-45bd-9c3f-8d28173b929c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:22.766333 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423615e-37c8-45bd-9c3f-8d28173b929c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:22.766700 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423615e-37c8-45bd-9c3f-8d28173b929c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:22.780737 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:22.829997 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4760e250-140c-4b90-96bd-f4da1479437e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:22.831707 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4760e250-140c-4b90-96bd-f4da1479437e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:22.888889 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6423615e-37c8-45bd-9c3f-8d28173b929c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:22.890876 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6423615e-37c8-45bd-9c3f-8d28173b929c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:23.024472 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 216/441] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:08:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 19:08:23.026297 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59139118-ee60-4dfd-b7f7-cc575ed14211 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:23.026845 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59139118-ee60-4dfd-b7f7-cc575ed14211 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:23.027352 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59139118-ee60-4dfd-b7f7-cc575ed14211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:23.041464 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:23.054460 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-59139118-ee60-4dfd-b7f7-cc575ed14211 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:23.058062 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-59139118-ee60-4dfd-b7f7-cc575ed14211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:23.075142 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 226/442] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:08:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 575 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 19:08:23.076218 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423615e-37c8-45bd-9c3f-8d28173b929c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:23.198020 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 217/443] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:08:23.199150 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59139118-ee60-4dfd-b7f7-cc575ed14211 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:23.225892 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:23.275806 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423615e-37c8-45bd-9c3f-8d28173b929c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:23.332319 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c0e81-6916-44fb-a2cf-8fc909cfa33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:23.333220 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c0e81-6916-44fb-a2cf-8fc909cfa33f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:23.333504 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c0e81-6916-44fb-a2cf-8fc909cfa33f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:23.333784 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c0e81-6916-44fb-a2cf-8fc909cfa33f None admin] PATH_INFO: `/v3/projects/0e408b6ba932474fb12e507cab566abd` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:23.335207 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c0e81-6916-44fb-a2cf-8fc909cfa33f None admin] RBAC: Authorizing `identity:get_project(project_id=0e408b6ba932474fb12e507cab566abd)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:23.336965 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c0e81-6916-44fb-a2cf-8fc909cfa33f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:23.339003 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 227/444] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:08:23 2022] GET /identity/v3/projects/0e408b6ba932474fb12e507cab566abd => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:23.339928 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c0e81-6916-44fb-a2cf-8fc909cfa33f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:23.384628 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59139118-ee60-4dfd-b7f7-cc575ed14211 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:23.413649 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:23.445533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5391e-a3cf-4788-9a8f-53cab1373c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:23.446577 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5391e-a3cf-4788-9a8f-53cab1373c23 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:23.447040 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5391e-a3cf-4788-9a8f-53cab1373c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:23.447480 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5391e-a3cf-4788-9a8f-53cab1373c23 None admin] PATH_INFO: `/v3/projects/0e408b6ba932474fb12e507cab566abd` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:23.448978 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f5391e-a3cf-4788-9a8f-53cab1373c23 None admin] RBAC: Authorizing `identity:get_project(project_id=0e408b6ba932474fb12e507cab566abd)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:23.453064 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f5391e-a3cf-4788-9a8f-53cab1373c23 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:23.455107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 218/445] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:08:23 2022] GET /identity/v3/projects/0e408b6ba932474fb12e507cab566abd => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:23.457321 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5391e-a3cf-4788-9a8f-53cab1373c23 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:23.513588 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c0e81-6916-44fb-a2cf-8fc909cfa33f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:23.574578 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:23.589902 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc637d5b-06e5-4f61-842a-f4366fff7137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:23.591986 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc637d5b-06e5-4f61-842a-f4366fff7137 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:23.592963 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc637d5b-06e5-4f61-842a-f4366fff7137 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:23.593510 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc637d5b-06e5-4f61-842a-f4366fff7137 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:23.607755 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5391e-a3cf-4788-9a8f-53cab1373c23 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:23.612185 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc637d5b-06e5-4f61-842a-f4366fff7137 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:08:23.621267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-cc637d5b-06e5-4f61-842a-f4366fff7137 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:08:23.621267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:23.621267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:23.621267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:23.621267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:23.621267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:23.621267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:23.621267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:23.621267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:23.621267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:23.621267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:23.621267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:23.621267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:08:23.621267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:08:23.621267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:08:23.621267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:08:23.623023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:08:23.623023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:08:23.623023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:08:23.623023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:08:23.623023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:23.623023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:23.623023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:23.623023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:23.623023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:23.623023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:23.623023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:23.623023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:23.623023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:23.623023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:23.623023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:23.623023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:23.624505 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:23.624505 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:23.624505 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:23.624505 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:23.624505 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:08:23.624505 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:08:23.624505 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:08:23.624505 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:08:23.624505 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:08:23.624505 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:08:23.624505 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:08:23.624505 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:23.624505 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 228/446] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Jul 25 19:08:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:23.624505 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc637d5b-06e5-4f61-842a-f4366fff7137 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:23.677756 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0f2f52-d6cb-4a84-9b0e-b81bdd5b6dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:23.678925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f2f52-d6cb-4a84-9b0e-b81bdd5b6dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:23.679304 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f2f52-d6cb-4a84-9b0e-b81bdd5b6dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:23.679730 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f2f52-d6cb-4a84-9b0e-b81bdd5b6dc8 None admin] PATH_INFO: `/v3/groups/7c8589271cc74ab88f7862a00a1915cb` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:23.681248 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0f2f52-d6cb-4a84-9b0e-b81bdd5b6dc8 None admin] RBAC: Authorizing `identity:get_group(group_id=7c8589271cc74ab88f7862a00a1915cb)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:23.682926 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0f2f52-d6cb-4a84-9b0e-b81bdd5b6dc8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:23.684935 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 219/447] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Jul 25 19:08:23 2022] GET /identity/v3/groups/7c8589271cc74ab88f7862a00a1915cb => generated 226 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:23.685779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0f2f52-d6cb-4a84-9b0e-b81bdd5b6dc8 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:23.808533 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc637d5b-06e5-4f61-842a-f4366fff7137 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:23.821254 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:23.844538 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0f2f52-d6cb-4a84-9b0e-b81bdd5b6dc8 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:23.859309 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c230c75-b049-4c6c-aa19-fc89e5af9ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:23.860261 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c230c75-b049-4c6c-aa19-fc89e5af9ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:23.860545 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c230c75-b049-4c6c-aa19-fc89e5af9ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:23.860901 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c230c75-b049-4c6c-aa19-fc89e5af9ce0 None admin] PATH_INFO: `/v3/groups/7c8589271cc74ab88f7862a00a1915cb` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:23.862277 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c230c75-b049-4c6c-aa19-fc89e5af9ce0 None admin] RBAC: Authorizing `identity:get_group(group_id=7c8589271cc74ab88f7862a00a1915cb)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:23.863823 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c230c75-b049-4c6c-aa19-fc89e5af9ce0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:23.865631 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 229/448] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Jul 25 19:08:23 2022] GET /identity/v3/groups/7c8589271cc74ab88f7862a00a1915cb => generated 226 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:23.865631 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c230c75-b049-4c6c-aa19-fc89e5af9ce0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:23.909977 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91fb73-4432-4023-996a-e05d7805ecac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:23.911188 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91fb73-4432-4023-996a-e05d7805ecac None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:23.911959 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91fb73-4432-4023-996a-e05d7805ecac None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:23.913232 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91fb73-4432-4023-996a-e05d7805ecac None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:23.914352 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91fb73-4432-4023-996a-e05d7805ecac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:23.914820 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91fb73-4432-4023-996a-e05d7805ecac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:23.916476 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91fb73-4432-4023-996a-e05d7805ecac None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:23.928733 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 220/449] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:08:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:23.929455 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91fb73-4432-4023-996a-e05d7805ecac None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:23.944151 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:24.035733 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c230c75-b049-4c6c-aa19-fc89e5af9ce0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:24.077937 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91fb73-4432-4023-996a-e05d7805ecac None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:24.083216 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a2a99-4953-4efa-999b-864a6899858a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:24.083681 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a2a99-4953-4efa-999b-864a6899858a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:24.083978 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a2a99-4953-4efa-999b-864a6899858a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:24.084288 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a2a99-4953-4efa-999b-864a6899858a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:24.085140 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a2a99-4953-4efa-999b-864a6899858a None admin] RBAC: Adding query filter params (group.id=7c8589271cc74ab88f7862a00a1915cb, role.id=c241bd9634e24bcbb1854c7621d19f81, scope.project.id=0e408b6ba932474fb12e507cab566abd) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:24.085459 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a2a99-4953-4efa-999b-864a6899858a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:24.086959 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a2a99-4953-4efa-999b-864a6899858a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:24.103755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 230/450] 10.222.0.18 () {56 vars in 1376 bytes} [Mon Jul 25 19:08:23 2022] GET /identity/v3/role_assignments?group.id=7c8589271cc74ab88f7862a00a1915cb&scope.project.id=0e408b6ba932474fb12e507cab566abd&role.id=c241bd9634e24bcbb1854c7621d19f81 => generated 611 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:24.103755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a2a99-4953-4efa-999b-864a6899858a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:24.147787 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed9559-1a36-46c2-947e-ba39872182bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:24.148847 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed9559-1a36-46c2-947e-ba39872182bd None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:24.149294 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed9559-1a36-46c2-947e-ba39872182bd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:24.149730 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed9559-1a36-46c2-947e-ba39872182bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:24.151204 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ed9559-1a36-46c2-947e-ba39872182bd None admin] RBAC: Adding query filter params (group.id=7c8589271cc74ab88f7862a00a1915cb, role.id=191e7e31dad148b7a0eb52d3afb5e50e, scope.project.id=0e408b6ba932474fb12e507cab566abd) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:24.151759 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ed9559-1a36-46c2-947e-ba39872182bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:24.155701 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ed9559-1a36-46c2-947e-ba39872182bd None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:24.160989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:24.175725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 221/451] 10.222.0.18 () {56 vars in 1376 bytes} [Mon Jul 25 19:08:23 2022] GET /identity/v3/role_assignments?group.id=7c8589271cc74ab88f7862a00a1915cb&scope.project.id=0e408b6ba932474fb12e507cab566abd&role.id=191e7e31dad148b7a0eb52d3afb5e50e => generated 611 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:24.273407 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a2a99-4953-4efa-999b-864a6899858a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:24.334622 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd996ed2-0b1c-4f68-9400-13c1d4c0e183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:24.335733 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd996ed2-0b1c-4f68-9400-13c1d4c0e183 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:24.336237 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd996ed2-0b1c-4f68-9400-13c1d4c0e183 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:24.336700 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd996ed2-0b1c-4f68-9400-13c1d4c0e183 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:24.470637 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd996ed2-0b1c-4f68-9400-13c1d4c0e183 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:24.473401 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd996ed2-0b1c-4f68-9400-13c1d4c0e183 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:24.548657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:24.657935 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 231/452] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:08:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 19:08:24.664372 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed9559-1a36-46c2-947e-ba39872182bd None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:24.777455 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:24.825071 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed9559-1a36-46c2-947e-ba39872182bd None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:24.868987 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e71277-8132-41d9-8e2c-1f07470f0807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:24.869881 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e71277-8132-41d9-8e2c-1f07470f0807 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:24.870169 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e71277-8132-41d9-8e2c-1f07470f0807 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:24.870441 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e71277-8132-41d9-8e2c-1f07470f0807 None admin] PATH_INFO: `/v3/projects/9565eeec7ac7457abc5ee445dbf84c18` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:24.871870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e71277-8132-41d9-8e2c-1f07470f0807 None admin] RBAC: Authorizing `identity:get_project(project_id=9565eeec7ac7457abc5ee445dbf84c18)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:24.874174 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e71277-8132-41d9-8e2c-1f07470f0807 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:24.876323 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 222/453] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:08:24 2022] GET /identity/v3/projects/9565eeec7ac7457abc5ee445dbf84c18 => generated 344 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:08:24.883829 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd996ed2-0b1c-4f68-9400-13c1d4c0e183 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:25.077851 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd996ed2-0b1c-4f68-9400-13c1d4c0e183 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:25.123992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101df680-a1c3-4738-99c0-966013210ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:25.125091 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101df680-a1c3-4738-99c0-966013210ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:25.125301 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101df680-a1c3-4738-99c0-966013210ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:25.125627 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101df680-a1c3-4738-99c0-966013210ae4 None admin] PATH_INFO: `/v3/groups/f7486e94158a4a698230dda9997c9d9f` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:25.127061 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101df680-a1c3-4738-99c0-966013210ae4 None admin] RBAC: Authorizing `identity:get_group(group_id=f7486e94158a4a698230dda9997c9d9f)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:25.128647 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101df680-a1c3-4738-99c0-966013210ae4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:25.130470 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 232/454] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Jul 25 19:08:24 2022] GET /identity/v3/groups/f7486e94158a4a698230dda9997c9d9f => generated 229 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:25.137021 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e71277-8132-41d9-8e2c-1f07470f0807 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:25.273544 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e71277-8132-41d9-8e2c-1f07470f0807 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:25.317669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf53e6-d1fd-4a26-855e-3df5a9a1e707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:25.318378 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf53e6-d1fd-4a26-855e-3df5a9a1e707 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:25.318664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf53e6-d1fd-4a26-855e-3df5a9a1e707 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:25.318959 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf53e6-d1fd-4a26-855e-3df5a9a1e707 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:25.319904 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf53e6-d1fd-4a26-855e-3df5a9a1e707 None admin] RBAC: Adding query filter params (group.id=f7486e94158a4a698230dda9997c9d9f, role.id=ed7fec18968241efa76a0ee769d96da9, scope.project.id=9565eeec7ac7457abc5ee445dbf84c18) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:25.320545 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf53e6-d1fd-4a26-855e-3df5a9a1e707 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:25.321805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf53e6-d1fd-4a26-855e-3df5a9a1e707 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:25.330240 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 223/455] 10.222.0.18 () {56 vars in 1376 bytes} [Mon Jul 25 19:08:25 2022] GET /identity/v3/role_assignments?group.id=f7486e94158a4a698230dda9997c9d9f&scope.project.id=9565eeec7ac7457abc5ee445dbf84c18&role.id=ed7fec18968241efa76a0ee769d96da9 => generated 611 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:08:28.721479 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27000fc7-6680-491b-8e82-905431cbd21c None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:28.722051 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27000fc7-6680-491b-8e82-905431cbd21c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:28.722051 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27000fc7-6680-491b-8e82-905431cbd21c None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:28.724299 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 233/456] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:08:28.733684 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f11a36-ca4d-4057-b26c-5f54c02f727a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:28.734444 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f11a36-ca4d-4057-b26c-5f54c02f727a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:28.734616 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f11a36-ca4d-4057-b26c-5f54c02f727a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:28.749974 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:28.768915 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e0f11a36-ca4d-4057-b26c-5f54c02f727a None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:28.770228 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e0f11a36-ca4d-4057-b26c-5f54c02f727a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:28.885451 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 224/457] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:08:28.965763 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27000fc7-6680-491b-8e82-905431cbd21c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:28.980949 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:29.240150 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27000fc7-6680-491b-8e82-905431cbd21c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:29.329880 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdcc2f6-0f82-44a7-b1b8-1fc43fa3e949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:29.331323 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdcc2f6-0f82-44a7-b1b8-1fc43fa3e949 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:29.331876 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdcc2f6-0f82-44a7-b1b8-1fc43fa3e949 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:29.332386 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdcc2f6-0f82-44a7-b1b8-1fc43fa3e949 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:29.363754 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdcc2f6-0f82-44a7-b1b8-1fc43fa3e949 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:29.366730 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdcc2f6-0f82-44a7-b1b8-1fc43fa3e949 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:29.413989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-0bdcc2f6-0f82-44a7-b1b8-1fc43fa3e949 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:29.413989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:29.413989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:29.413989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:29.413989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:29.413989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:29.413989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:29.413989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:29.413989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:29.413989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:29.413989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:29.413989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:29.413989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:08:29.413989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:08:29.413989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:08:29.415598 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:08:29.415598 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:29.415598 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:29.415598 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:29.415598 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:29.415598 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:29.415598 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:29.415598 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:29.415598 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:29.415598 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:29.415598 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:29.415598 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:29.415598 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:29.415598 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:29.415598 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:29.415598 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:29.415598 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:29.417300 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:08:29.417300 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:08:29.417300 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:29.417300 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:29.417300 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 234/458] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:08:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 451 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:29.424049 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f11a36-ca4d-4057-b26c-5f54c02f727a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:29.576007 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf33428-3576-45d0-bcb1-2c765547c876 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:29.577096 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf33428-3576-45d0-bcb1-2c765547c876 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:29.577917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf33428-3576-45d0-bcb1-2c765547c876 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:29.587562 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 235/459] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:08:29.591722 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d1039-106b-4252-ab03-44ae8e35e73a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:29.592629 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d1039-106b-4252-ab03-44ae8e35e73a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:29.593553 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d1039-106b-4252-ab03-44ae8e35e73a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:29.648723 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f11a36-ca4d-4057-b26c-5f54c02f727a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:29.659453 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-851d1039-106b-4252-ab03-44ae8e35e73a None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:29.661721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-851d1039-106b-4252-ab03-44ae8e35e73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:29.774699 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:29.795856 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:29.804956 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639d74c-d24f-4a78-8967-493dee7f0460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:29.806174 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639d74c-d24f-4a78-8967-493dee7f0460 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:29.806687 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639d74c-d24f-4a78-8967-493dee7f0460 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:29.807268 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639d74c-d24f-4a78-8967-493dee7f0460 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:29.808750 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639d74c-d24f-4a78-8967-493dee7f0460 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:29.809513 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639d74c-d24f-4a78-8967-493dee7f0460 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:29.812881 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639d74c-d24f-4a78-8967-493dee7f0460 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:29.816634 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 236/460] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 19:08:29.847223 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 225/461] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 424 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:29.857439 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d1039-106b-4252-ab03-44ae8e35e73a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:29.932228 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639d74c-d24f-4a78-8967-493dee7f0460 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:30.113234 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d1039-106b-4252-ab03-44ae8e35e73a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:30.148700 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639d74c-d24f-4a78-8967-493dee7f0460 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:30.162731 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7bd052-a311-4c63-88cb-afd374b625a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:30.164234 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7bd052-a311-4c63-88cb-afd374b625a9 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:30.164594 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7bd052-a311-4c63-88cb-afd374b625a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:30.165158 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7bd052-a311-4c63-88cb-afd374b625a9 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:30.166140 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7bd052-a311-4c63-88cb-afd374b625a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:30.167773 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7bd052-a311-4c63-88cb-afd374b625a9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:30.211575 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 237/462] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Jul 25 19:08:29 2022] POST /identity/v3/users => generated 246 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:08:30.211575 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af1812-3bed-4aa6-87ed-5a8e55ee93a8 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:30.211575 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af1812-3bed-4aa6-87ed-5a8e55ee93a8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:30.211575 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af1812-3bed-4aa6-87ed-5a8e55ee93a8 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:30.211575 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 238/463] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:08:30.211575 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475b9b7-fa5a-4d9b-8cce-6b093d2eee42 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:30.211575 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475b9b7-fa5a-4d9b-8cce-6b093d2eee42 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:30.212621 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475b9b7-fa5a-4d9b-8cce-6b093d2eee42 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:30.218373 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 239/464] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:30 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:08:30.220350 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83295cf-3a6d-4682-9126-9ea7de14aa98 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:30.220549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a93b65d-8711-4f4a-b301-54dc2e4dd44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:30.220745 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83295cf-3a6d-4682-9126-9ea7de14aa98 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:30.221050 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83295cf-3a6d-4682-9126-9ea7de14aa98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:30.222188 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a93b65d-8711-4f4a-b301-54dc2e4dd44a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:30.222789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a93b65d-8711-4f4a-b301-54dc2e4dd44a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:30.223277 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a93b65d-8711-4f4a-b301-54dc2e4dd44a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:30.249868 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a93b65d-8711-4f4a-b301-54dc2e4dd44a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:30.252423 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a93b65d-8711-4f4a-b301-54dc2e4dd44a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:30.252626 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e83295cf-3a6d-4682-9126-9ea7de14aa98 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:30.253871 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e83295cf-3a6d-4682-9126-9ea7de14aa98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:30.305839 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-9a93b65d-8711-4f4a-b301-54dc2e4dd44a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:30.305839 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:30.305839 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:30.305839 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:30.305839 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:30.305839 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:30.305839 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:30.305839 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:30.305839 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:30.305839 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:30.305839 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:30.305839 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:30.305839 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:08:30.305839 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:08:30.305839 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:08:30.307432 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:08:30.307432 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:30.307432 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:30.307432 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:30.307432 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:30.307432 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:30.307432 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:30.307432 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:30.307432 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:30.307432 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:30.307432 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:30.307432 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:30.307432 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:30.307432 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:30.307432 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:30.307432 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:30.307432 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:30.309736 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:08:30.309736 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:08:30.309736 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:30.309736 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:30.309736 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 226/465] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:08:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:30.311043 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd6f36-504a-4116-ae5d-c94d0e1fca65 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:30.312116 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd6f36-504a-4116-ae5d-c94d0e1fca65 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:30.312476 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd6f36-504a-4116-ae5d-c94d0e1fca65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:30.347222 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:30.360127 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-23fd6f36-504a-4116-ae5d-c94d0e1fca65 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:30.361425 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-23fd6f36-504a-4116-ae5d-c94d0e1fca65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:30.404665 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 240/466] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:08:30.406741 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83295cf-3a6d-4682-9126-9ea7de14aa98 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:30.436053 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:30.462621 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 227/467] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:08:30.465026 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84e9d1-7824-4dd8-a1a9-ec3739d9d8f0 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:30.465684 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84e9d1-7824-4dd8-a1a9-ec3739d9d8f0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:30.466052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84e9d1-7824-4dd8-a1a9-ec3739d9d8f0 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:30.469660 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 228/468] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:08:30.471147 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e84e9d1-7824-4dd8-a1a9-ec3739d9d8f0 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:30.619790 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83295cf-3a6d-4682-9126-9ea7de14aa98 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:30.660476 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e84e9d1-7824-4dd8-a1a9-ec3739d9d8f0 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:30.690663 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cbc27b-0f78-4398-847e-b9fe4b79445a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:30.691906 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cbc27b-0f78-4398-847e-b9fe4b79445a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:30.692389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cbc27b-0f78-4398-847e-b9fe4b79445a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:30.692969 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cbc27b-0f78-4398-847e-b9fe4b79445a None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:30.694308 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cbc27b-0f78-4398-847e-b9fe4b79445a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:30.695212 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cbc27b-0f78-4398-847e-b9fe4b79445a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:30.697502 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cbc27b-0f78-4398-847e-b9fe4b79445a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:30.719631 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4281c64-23a5-473e-966b-cff7b406c61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:30.720840 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4281c64-23a5-473e-966b-cff7b406c61a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:30.721248 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4281c64-23a5-473e-966b-cff7b406c61a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:30.721644 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4281c64-23a5-473e-966b-cff7b406c61a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:30.740250 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 241/469] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:30.741525 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9f3cb-a644-4daf-a383-7d520bb5669e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:30.742904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9f3cb-a644-4daf-a383-7d520bb5669e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:30.742904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9f3cb-a644-4daf-a383-7d520bb5669e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:30.749818 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4281c64-23a5-473e-966b-cff7b406c61a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:30.752315 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4281c64-23a5-473e-966b-cff7b406c61a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:30.769372 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1ce9f3cb-a644-4daf-a383-7d520bb5669e None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:30.771347 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1ce9f3cb-a644-4daf-a383-7d520bb5669e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:30.779125 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b4281c64-23a5-473e-966b-cff7b406c61a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:30.779125 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:30.779125 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:30.779125 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:30.779125 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:30.779125 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:30.779125 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:30.779125 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:30.779125 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:30.779125 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:30.779125 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:30.779125 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:30.779125 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:08:30.779125 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:08:30.779125 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:08:30.781790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:08:30.781790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:30.781790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:30.781790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:30.781790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:30.781790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:30.781790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:30.781790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:30.781790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:30.781790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:30.781790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:30.781790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:30.781790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:30.781790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:30.781790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:30.781790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:30.781790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:30.783880 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:08:30.783880 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:08:30.783880 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:30.783880 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:30.783880 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 229/470] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:08:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:30.787145 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4281c64-23a5-473e-966b-cff7b406c61a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:30.831876 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:30.844305 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:30.897391 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 242/471] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:08:30.898230 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9f3cb-a644-4daf-a383-7d520bb5669e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:30.983459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4281c64-23a5-473e-966b-cff7b406c61a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:31.048505 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f4c42-d512-451e-a5c4-85549819e46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:31.049763 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f4c42-d512-451e-a5c4-85549819e46f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:31.050169 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f4c42-d512-451e-a5c4-85549819e46f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:31.050453 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923f4c42-d512-451e-a5c4-85549819e46f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:31.065373 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923f4c42-d512-451e-a5c4-85549819e46f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:31.067655 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923f4c42-d512-451e-a5c4-85549819e46f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:31.098403 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-923f4c42-d512-451e-a5c4-85549819e46f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:31.098403 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:31.098403 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:31.098403 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:31.098403 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:31.098403 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:31.098403 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:31.098403 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:31.098403 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:31.098403 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:31.098403 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:31.098403 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:31.098403 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:08:31.098403 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:08:31.098403 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:08:31.100388 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:08:31.100388 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:31.100388 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:31.100388 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:31.100388 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:31.100388 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:31.100388 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:31.100388 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:31.100388 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:31.100388 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:31.100388 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:31.100388 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:31.100388 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:31.100388 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:31.100388 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:31.100388 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:31.100388 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:31.102257 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:08:31.102257 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:08:31.102257 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:31.102257 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:31.106669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 230/472] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:08:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:31.108388 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f4c42-d512-451e-a5c4-85549819e46f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:31.177481 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9f3cb-a644-4daf-a383-7d520bb5669e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:31.233425 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575d7d3-83b6-4946-8197-fa90c33dd2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:31.234187 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575d7d3-83b6-4946-8197-fa90c33dd2ab None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:31.234418 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575d7d3-83b6-4946-8197-fa90c33dd2ab None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:31.234702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575d7d3-83b6-4946-8197-fa90c33dd2ab None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:31.235444 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575d7d3-83b6-4946-8197-fa90c33dd2ab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:31.237015 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575d7d3-83b6-4946-8197-fa90c33dd2ab None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:31.246207 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:31.280884 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923f4c42-d512-451e-a5c4-85549819e46f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:31.291543 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 243/473] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Jul 25 19:08:30 2022] POST /identity/v3/users => generated 248 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:08:31.291543 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575d7d3-83b6-4946-8197-fa90c33dd2ab None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:31.355143 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:31.359202 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad297cf-b82c-4c22-b64a-6d37120bdb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:31.361344 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad297cf-b82c-4c22-b64a-6d37120bdb14 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:31.361952 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad297cf-b82c-4c22-b64a-6d37120bdb14 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:31.362455 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad297cf-b82c-4c22-b64a-6d37120bdb14 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:31.364638 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad297cf-b82c-4c22-b64a-6d37120bdb14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:31.365658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad297cf-b82c-4c22-b64a-6d37120bdb14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:31.368998 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad297cf-b82c-4c22-b64a-6d37120bdb14 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:31.404225 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 231/474] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:31.406100 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad297cf-b82c-4c22-b64a-6d37120bdb14 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:31.495361 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575d7d3-83b6-4946-8197-fa90c33dd2ab None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:31.565492 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ddfa9-0da6-4a49-a12e-60afd3a848e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:31.566359 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ddfa9-0da6-4a49-a12e-60afd3a848e6 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:31.566854 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ddfa9-0da6-4a49-a12e-60afd3a848e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:31.567329 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ddfa9-0da6-4a49-a12e-60afd3a848e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:31.568476 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987ddfa9-0da6-4a49-a12e-60afd3a848e6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:31.570246 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987ddfa9-0da6-4a49-a12e-60afd3a848e6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:31.587846 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.sql.core [None req-987ddfa9-0da6-4a49-a12e-60afd3a848e6 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 25 19:08:31.587846 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 25 19:08:31.587846 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [parameters: {'id': '438f8fa580e244c188ab721b37b1437e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 25 19:08:31.587846 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80237) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 25 19:08:31.620860 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-987ddfa9-0da6-4a49-a12e-60afd3a848e6 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 25 19:08:31.620860 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:31.620860 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 19:08:31.620860 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 19:08:31.620860 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 19:08:31.620860 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 19:08:31.620860 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 19:08:31.620860 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 19:08:31.620860 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 19:08:31.620860 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application conn.query(q) Jul 25 19:08:31.620860 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 19:08:31.620860 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 19:08:31.620860 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 19:08:31.620860 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result.read() Jul 25 19:08:31.620860 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 19:08:31.623215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 19:08:31.623215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 19:08:31.623215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 19:08:31.623215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 19:08:31.623215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 19:08:31.623215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 19:08:31.623215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 19:08:31.623215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 25 19:08:31.623215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:31.623215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 25 19:08:31.623215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:31.623215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:31.623215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 25 19:08:31.623215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 25 19:08:31.623215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 25 19:08:31.623215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return ref.to_dict() Jul 25 19:08:31.623215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 19:08:31.623215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application next(self.gen) Jul 25 19:08:31.625486 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 25 19:08:31.625486 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application yield resource Jul 25 19:08:31.625486 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 19:08:31.625486 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application next(self.gen) Jul 25 19:08:31.625486 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 25 19:08:31.625486 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.session.rollback() Jul 25 19:08:31.625486 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 25 19:08:31.625486 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.force_reraise() Jul 25 19:08:31.625486 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 25 19:08:31.625486 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise self.value Jul 25 19:08:31.625486 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 25 19:08:31.625486 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 25 19:08:31.625486 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 25 19:08:31.625486 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application session.commit() Jul 25 19:08:31.625486 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 25 19:08:31.625486 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 25 19:08:31.625486 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 25 19:08:31.627452 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._prepare_impl() Jul 25 19:08:31.627452 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 25 19:08:31.627452 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.session.flush() Jul 25 19:08:31.627452 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 25 19:08:31.627452 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._flush(objects) Jul 25 19:08:31.627452 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 25 19:08:31.627452 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 25 19:08:31.627452 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 25 19:08:31.627452 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application compat.raise_( Jul 25 19:08:31.627452 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 19:08:31.627452 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception Jul 25 19:08:31.627452 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 25 19:08:31.627452 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application flush_context.execute() Jul 25 19:08:31.627452 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 25 19:08:31.627452 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rec.execute(self) Jul 25 19:08:31.627452 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 25 19:08:31.627452 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 25 19:08:31.632343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 25 19:08:31.632343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 25 19:08:31.632343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 25 19:08:31.632343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 25 19:08:31.632343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 25 19:08:31.632343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 25 19:08:31.632343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 25 19:08:31.632343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 25 19:08:31.632343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 25 19:08:31.632343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 25 19:08:31.632343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 25 19:08:31.632343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 25 19:08:31.632343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 25 19:08:31.632343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 25 19:08:31.632343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 19:08:31.632343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception Jul 25 19:08:31.635053 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 19:08:31.635053 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 19:08:31.635053 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 19:08:31.635053 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 19:08:31.635053 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 19:08:31.635053 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 19:08:31.635053 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 19:08:31.635053 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application conn.query(q) Jul 25 19:08:31.635053 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 19:08:31.635053 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 19:08:31.635053 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 19:08:31.635053 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result.read() Jul 25 19:08:31.635053 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 19:08:31.635053 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 19:08:31.635053 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 19:08:31.635053 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 19:08:31.640228 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 19:08:31.640228 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 19:08:31.640228 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 19:08:31.640228 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 19:08:31.640228 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 25 19:08:31.640228 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 25 19:08:31.640228 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [parameters: {'id': '438f8fa580e244c188ab721b37b1437e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 25 19:08:31.640228 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 25 19:08:31.640228 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:31.640228 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 19:08:31.640228 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:31.640228 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:31.640228 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:31.640228 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:31.640228 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:31.640228 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:31.642034 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:31.642034 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:31.642034 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:31.642034 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:31.642034 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:31.642034 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:31.642034 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 25 19:08:31.642034 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 25 19:08:31.642034 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:31.642034 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:31.642034 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 25 19:08:31.642034 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 25 19:08:31.642034 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 25 19:08:31.642034 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 25 19:08:31.642034 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 25 19:08:31.642034 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:31.648215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 244/475] 10.222.0.18 () {60 vars in 1131 bytes} [Mon Jul 25 19:08:31 2022] POST /identity/v3/roles => generated 144 bytes in 340 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:31.648215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ddfa9-0da6-4a49-a12e-60afd3a848e6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:31.664336 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad297cf-b82c-4c22-b64a-6d37120bdb14 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:31.749269 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea0e7e3-0c2e-48a2-aae4-82f1f259ab6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:31.750557 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0e7e3-0c2e-48a2-aae4-82f1f259ab6c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:31.751168 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0e7e3-0c2e-48a2-aae4-82f1f259ab6c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:31.751746 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0e7e3-0c2e-48a2-aae4-82f1f259ab6c None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:31.753252 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea0e7e3-0c2e-48a2-aae4-82f1f259ab6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:31.753760 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea0e7e3-0c2e-48a2-aae4-82f1f259ab6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:31.757014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea0e7e3-0c2e-48a2-aae4-82f1f259ab6c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:31.773183 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:31.786051 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 232/476] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:31.787811 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea0e7e3-0c2e-48a2-aae4-82f1f259ab6c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:31.839863 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:31.878594 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ddfa9-0da6-4a49-a12e-60afd3a848e6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:31.945534 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e22c5a-1b27-46a7-97ad-2612e435afbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:31.946565 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e22c5a-1b27-46a7-97ad-2612e435afbc None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:31.946981 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e22c5a-1b27-46a7-97ad-2612e435afbc None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:31.947395 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e22c5a-1b27-46a7-97ad-2612e435afbc None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:31.948549 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e22c5a-1b27-46a7-97ad-2612e435afbc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:31.951334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e22c5a-1b27-46a7-97ad-2612e435afbc None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:31.989681 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 245/477] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Jul 25 19:08:31 2022] POST /identity/v3/users => generated 249 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:08:31.990791 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e22c5a-1b27-46a7-97ad-2612e435afbc None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:32.034777 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea0e7e3-0c2e-48a2-aae4-82f1f259ab6c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:32.145791 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed07512-46ca-4659-a5ca-276f761319b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:32.147266 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed07512-46ca-4659-a5ca-276f761319b7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:32.147784 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed07512-46ca-4659-a5ca-276f761319b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:32.148293 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed07512-46ca-4659-a5ca-276f761319b7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:32.177157 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed07512-46ca-4659-a5ca-276f761319b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:08:32.211470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-eed07512-46ca-4659-a5ca-276f761319b7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 19:08:32.211470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:32.211470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:32.211470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:32.211470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:32.211470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:32.211470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:32.211470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:32.211470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:32.211470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:32.211470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:32.211470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:32.211470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:08:32.211470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:08:32.211470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:08:32.211470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:08:32.212700 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:08:32.212700 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:08:32.212700 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:08:32.212700 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:08:32.212700 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:32.212700 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:32.212700 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:32.212700 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:32.212700 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:32.212700 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:32.212700 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:32.212700 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:32.212700 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:32.212700 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:32.212700 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:32.212700 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:32.213569 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:32.213569 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:32.213569 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:32.213569 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:32.213569 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:08:32.213569 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:08:32.213569 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:08:32.213569 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:08:32.213569 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:08:32.213569 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:08:32.213569 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 19:08:32.213569 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:32.214237 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 233/478] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Jul 25 19:08:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 427 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:32.225952 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed07512-46ca-4659-a5ca-276f761319b7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:32.277100 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e22c5a-1b27-46a7-97ad-2612e435afbc None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:32.360185 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d127447a-1f84-48e5-9ee5-fef0fa2a4e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:32.363260 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127447a-1f84-48e5-9ee5-fef0fa2a4e77 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:32.364389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127447a-1f84-48e5-9ee5-fef0fa2a4e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:32.364908 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d127447a-1f84-48e5-9ee5-fef0fa2a4e77 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:32.366021 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d127447a-1f84-48e5-9ee5-fef0fa2a4e77 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:32.368746 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d127447a-1f84-48e5-9ee5-fef0fa2a4e77 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:32.394280 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:32.396879 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:32.410343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 246/479] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Jul 25 19:08:31 2022] POST /identity/v3/users => generated 248 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:08:32.517318 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed07512-46ca-4659-a5ca-276f761319b7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:32.613860 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31db423-7b75-413f-872b-63738dff062b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:32.617166 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31db423-7b75-413f-872b-63738dff062b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:32.618238 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31db423-7b75-413f-872b-63738dff062b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:32.621911 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31db423-7b75-413f-872b-63738dff062b None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:32.624136 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31db423-7b75-413f-872b-63738dff062b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:32.625109 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31db423-7b75-413f-872b-63738dff062b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:32.629730 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31db423-7b75-413f-872b-63738dff062b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:32.655853 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 234/480] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:08:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 431 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:08:33.692610 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15111d-2263-44f4-89ff-f4f43635620a None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:33.693453 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15111d-2263-44f4-89ff-f4f43635620a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:33.693453 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15111d-2263-44f4-89ff-f4f43635620a None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:33.695088 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 247/481] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:08:33.702331 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5905f4-7087-431a-80b5-62a572e77392 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:33.703143 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5905f4-7087-431a-80b5-62a572e77392 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:33.703143 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5905f4-7087-431a-80b5-62a572e77392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:33.738692 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-cc5905f4-7087-431a-80b5-62a572e77392 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:33.740525 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cc5905f4-7087-431a-80b5-62a572e77392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:33.954822 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 235/482] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 19:08:34.004136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15111d-2263-44f4-89ff-f4f43635620a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:34.201941 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15111d-2263-44f4-89ff-f4f43635620a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:34.262807 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2fb83-e925-47db-98d9-a7999bda202d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:34.263642 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2fb83-e925-47db-98d9-a7999bda202d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:34.268309 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2fb83-e925-47db-98d9-a7999bda202d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:34.268713 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2fb83-e925-47db-98d9-a7999bda202d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:34.291380 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f2fb83-e925-47db-98d9-a7999bda202d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:08:34.303742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-04f2fb83-e925-47db-98d9-a7999bda202d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:08:34.303742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:34.303742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:34.303742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:34.303742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:34.303742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:34.303742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:34.303742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:34.303742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:34.303742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:34.303742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:34.303742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:34.303742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:08:34.303742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:08:34.303742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:08:34.303742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:08:34.305538 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:08:34.305538 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:08:34.305538 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:08:34.305538 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:08:34.305538 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:34.305538 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:34.305538 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:34.305538 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:34.305538 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:34.305538 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:34.305538 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:34.305538 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:34.305538 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:34.305538 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:34.305538 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:34.305538 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:34.307076 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:34.307076 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:34.307076 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:34.307076 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:34.307076 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:08:34.307076 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:08:34.307076 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:08:34.307076 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:08:34.307076 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:08:34.307076 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:08:34.307076 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:08:34.307076 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:34.307076 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 248/483] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Jul 25 19:08:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:34.313364 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5905f4-7087-431a-80b5-62a572e77392 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:34.464279 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2b621-7a05-409c-9094-44bbc6de3e91 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:34.464672 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2b621-7a05-409c-9094-44bbc6de3e91 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:34.465008 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2b621-7a05-409c-9094-44bbc6de3e91 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:34.467291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 249/484] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:08:34.469802 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:34.475394 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7023aa-413d-4fb3-a65b-ecf08e9f5124 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:34.475756 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7023aa-413d-4fb3-a65b-ecf08e9f5124 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:34.476092 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7023aa-413d-4fb3-a65b-ecf08e9f5124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:34.516796 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4e7023aa-413d-4fb3-a65b-ecf08e9f5124 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:34.518681 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4e7023aa-413d-4fb3-a65b-ecf08e9f5124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:34.613835 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5905f4-7087-431a-80b5-62a572e77392 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:34.702701 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:34.705738 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd430264-d6e9-47c2-b960-08bbf6a18b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:34.707535 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd430264-d6e9-47c2-b960-08bbf6a18b06 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:34.708179 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd430264-d6e9-47c2-b960-08bbf6a18b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:34.708822 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd430264-d6e9-47c2-b960-08bbf6a18b06 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:34.709921 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd430264-d6e9-47c2-b960-08bbf6a18b06 None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:34.711986 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd430264-d6e9-47c2-b960-08bbf6a18b06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:34.714131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd430264-d6e9-47c2-b960-08bbf6a18b06 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:34.734608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 236/485] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Jul 25 19:08:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:34.743747 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd430264-d6e9-47c2-b960-08bbf6a18b06 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:34.761832 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 250/486] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 19:08:34.818708 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7023aa-413d-4fb3-a65b-ecf08e9f5124 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:34.902851 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd430264-d6e9-47c2-b960-08bbf6a18b06 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:34.929832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:34.976181 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6db70-1859-46a6-8694-ef4ae0ed2ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:34.977017 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6db70-1859-46a6-8694-ef4ae0ed2ece None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:34.977311 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6db70-1859-46a6-8694-ef4ae0ed2ece None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:34.977593 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6db70-1859-46a6-8694-ef4ae0ed2ece None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:35.016679 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f6db70-1859-46a6-8694-ef4ae0ed2ece None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:35.018470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f6db70-1859-46a6-8694-ef4ae0ed2ece None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:35.056290 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7023aa-413d-4fb3-a65b-ecf08e9f5124 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:35.069604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b0f6db70-1859-46a6-8694-ef4ae0ed2ece None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:35.069604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:35.069604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:35.069604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:35.069604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:35.069604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:35.069604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:35.069604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:35.069604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:35.069604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:35.069604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:35.069604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:35.069604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:08:35.069604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:08:35.069604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:08:35.071410 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:08:35.071410 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:35.071410 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:35.071410 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:35.071410 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:35.071410 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:35.071410 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:35.071410 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:35.071410 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:35.071410 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:35.071410 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:35.071410 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:35.071410 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:35.071410 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:35.071410 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:35.071410 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:35.071410 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:35.073975 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:08:35.073975 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:08:35.073975 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:35.073975 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:35.073975 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 237/487] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:08:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:35.084533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6db70-1859-46a6-8694-ef4ae0ed2ece None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:35.128877 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a7e37-768c-4aa7-9075-04c42ac5cae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:35.129716 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a7e37-768c-4aa7-9075-04c42ac5cae5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:35.130092 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a7e37-768c-4aa7-9075-04c42ac5cae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:35.130373 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a7e37-768c-4aa7-9075-04c42ac5cae5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:35.158933 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145a7e37-768c-4aa7-9075-04c42ac5cae5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:08:35.167435 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-145a7e37-768c-4aa7-9075-04c42ac5cae5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:08:35.167435 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:35.167435 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:35.167435 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:35.167435 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:35.167435 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:35.167435 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:35.167435 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:35.167435 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:35.167435 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:35.167435 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:35.167435 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:35.167435 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:08:35.167435 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:08:35.167435 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:08:35.167435 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:08:35.168636 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:08:35.168636 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:08:35.168636 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:08:35.168636 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:08:35.168636 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:35.168636 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:35.168636 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:35.168636 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:35.168636 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:35.168636 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:35.168636 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:35.168636 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:35.168636 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:35.168636 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:35.168636 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:35.168636 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:35.170152 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:35.170152 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:35.170152 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:35.170152 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:35.170152 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:08:35.170152 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:08:35.170152 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:08:35.170152 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:08:35.170152 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:08:35.170152 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:08:35.170152 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:08:35.170152 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:35.170152 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 251/488] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Jul 25 19:08:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:08:35.177309 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a7e37-768c-4aa7-9075-04c42ac5cae5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:35.334784 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:35.436420 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6db70-1859-46a6-8694-ef4ae0ed2ece None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:35.522125 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a7e37-768c-4aa7-9075-04c42ac5cae5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:35.573338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0625301f-153a-471f-93ac-97478025aab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:35.574548 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625301f-153a-471f-93ac-97478025aab8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:35.576021 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625301f-153a-471f-93ac-97478025aab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:35.576464 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625301f-153a-471f-93ac-97478025aab8 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:35.577359 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0625301f-153a-471f-93ac-97478025aab8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:35.577684 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0625301f-153a-471f-93ac-97478025aab8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:35.579157 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0625301f-153a-471f-93ac-97478025aab8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:35.612538 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 238/489] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:08:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 533 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:35.614313 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef4003-92cc-4e62-98df-81761558686b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:35.615432 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef4003-92cc-4e62-98df-81761558686b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:35.615961 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef4003-92cc-4e62-98df-81761558686b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:35.616679 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef4003-92cc-4e62-98df-81761558686b None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:35.618146 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ef4003-92cc-4e62-98df-81761558686b None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:35.618655 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ef4003-92cc-4e62-98df-81761558686b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:35.619131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0625301f-153a-471f-93ac-97478025aab8 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:35.621170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ef4003-92cc-4e62-98df-81761558686b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:35.638633 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 252/490] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Jul 25 19:08:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 462 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:08:35.647074 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef4003-92cc-4e62-98df-81761558686b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:35.664855 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:35.801576 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0625301f-153a-471f-93ac-97478025aab8 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:35.874214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ae82a-4ddd-4a35-b8e7-15d8177f8833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:35.876032 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ae82a-4ddd-4a35-b8e7-15d8177f8833 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:35.877177 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ae82a-4ddd-4a35-b8e7-15d8177f8833 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:35.877585 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ae82a-4ddd-4a35-b8e7-15d8177f8833 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:35.899339 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7ae82a-4ddd-4a35-b8e7-15d8177f8833 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:35.901629 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7ae82a-4ddd-4a35-b8e7-15d8177f8833 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:35.949058 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-ce7ae82a-4ddd-4a35-b8e7-15d8177f8833 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 19:08:35.949058 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:35.949058 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:35.949058 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:35.949058 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:35.949058 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:35.949058 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:35.949058 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:35.949058 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:35.949058 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:35.949058 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:35.949058 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:35.949058 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:08:35.949058 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:08:35.949058 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:08:35.949058 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:08:35.951402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:35.951402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:35.951402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:08:35.951402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:08:35.951402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:08:35.951402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:08:35.951402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:35.951402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:35.951402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:35.951402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:35.951402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:35.951402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:35.951402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:35.951402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:35.951402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:35.951402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:35.951402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:35.951402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:35.953311 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:35.953311 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:35.953311 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:08:35.953311 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:08:35.953311 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:08:35.953311 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:08:35.953311 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:08:35.953311 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:08:35.953311 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 19:08:35.953311 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:35.957835 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 239/491] 10.222.0.18 () {56 vars in 1093 bytes} [Mon Jul 25 19:08:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:08:35.969668 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ae82a-4ddd-4a35-b8e7-15d8177f8833 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:36.063090 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef4003-92cc-4e62-98df-81761558686b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:36.086921 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:36.132807 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896afb10-1652-4b8c-b469-8d94f66b7da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:36.134530 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896afb10-1652-4b8c-b469-8d94f66b7da7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:36.135252 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896afb10-1652-4b8c-b469-8d94f66b7da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:36.135991 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896afb10-1652-4b8c-b469-8d94f66b7da7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:36.158650 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896afb10-1652-4b8c-b469-8d94f66b7da7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:36.162065 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896afb10-1652-4b8c-b469-8d94f66b7da7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:36.208066 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-896afb10-1652-4b8c-b469-8d94f66b7da7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:36.208066 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:36.208066 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:36.208066 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:36.208066 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:36.208066 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:36.208066 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:36.208066 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:36.208066 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:36.208066 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:36.208066 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:36.208066 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:36.208066 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:08:36.208066 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:08:36.208066 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:08:36.209441 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:08:36.209441 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:36.209441 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:36.209441 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:36.209441 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:36.209441 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:36.209441 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:36.209441 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:36.209441 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:36.209441 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:36.209441 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:36.209441 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:36.209441 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:36.209441 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:36.209441 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:36.209441 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:36.209441 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:36.211194 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:08:36.211194 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:08:36.211194 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:36.211194 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:36.218697 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 253/492] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:08:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 573 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:36.227185 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896afb10-1652-4b8c-b469-8d94f66b7da7 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:36.268144 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:36.307445 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ae82a-4ddd-4a35-b8e7-15d8177f8833 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:36.362116 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e15db-3f99-4ba0-81b7-58579475cc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:36.363422 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e15db-3f99-4ba0-81b7-58579475cc67 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:36.364271 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e15db-3f99-4ba0-81b7-58579475cc67 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:36.364812 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e15db-3f99-4ba0-81b7-58579475cc67 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:36.366543 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39e15db-3f99-4ba0-81b7-58579475cc67 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:36.367133 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39e15db-3f99-4ba0-81b7-58579475cc67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:36.369244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39e15db-3f99-4ba0-81b7-58579475cc67 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:36.427544 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 240/493] 10.222.0.18 () {56 vars in 1133 bytes} [Mon Jul 25 19:08:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 460 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:36.434786 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e15db-3f99-4ba0-81b7-58579475cc67 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:36.533598 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896afb10-1652-4b8c-b469-8d94f66b7da7 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:36.579546 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab47079-4378-407d-a74d-cacf24fcaf97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:36.580748 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab47079-4378-407d-a74d-cacf24fcaf97 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:36.581268 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab47079-4378-407d-a74d-cacf24fcaf97 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:36.581850 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab47079-4378-407d-a74d-cacf24fcaf97 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:36.582969 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab47079-4378-407d-a74d-cacf24fcaf97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:36.583619 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab47079-4378-407d-a74d-cacf24fcaf97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:36.585485 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab47079-4378-407d-a74d-cacf24fcaf97 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:36.602301 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 254/494] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:08:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:08:36.604496 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f87353-f2ec-4cf8-aaba-3849476c4ef9 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:36.605088 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f87353-f2ec-4cf8-aaba-3849476c4ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:36.605686 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f87353-f2ec-4cf8-aaba-3849476c4ef9 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:36.608549 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 255/495] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:08:36.609943 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f87353-f2ec-4cf8-aaba-3849476c4ef9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:36.636252 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e15db-3f99-4ba0-81b7-58579475cc67 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:36.644646 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:36.711561 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50827ed2-00d2-4775-a21e-06408489307b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:36.712590 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50827ed2-00d2-4775-a21e-06408489307b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:36.713078 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50827ed2-00d2-4775-a21e-06408489307b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:36.713561 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50827ed2-00d2-4775-a21e-06408489307b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:36.737114 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:36.769568 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f87353-f2ec-4cf8-aaba-3849476c4ef9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:36.842180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50827ed2-00d2-4775-a21e-06408489307b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:36.843813 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50827ed2-00d2-4775-a21e-06408489307b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:36.855440 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e13a7-6ef6-4965-8c9d-ec4a3748c79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:36.856722 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e13a7-6ef6-4965-8c9d-ec4a3748c79c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:36.857139 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e13a7-6ef6-4965-8c9d-ec4a3748c79c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:36.857541 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e13a7-6ef6-4965-8c9d-ec4a3748c79c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:36.871349 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e13a7-6ef6-4965-8c9d-ec4a3748c79c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:36.875039 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e13a7-6ef6-4965-8c9d-ec4a3748c79c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:36.902863 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-c03e13a7-6ef6-4965-8c9d-ec4a3748c79c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 25 19:08:36.902863 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:36.902863 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:36.902863 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:36.902863 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:36.902863 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:36.902863 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:36.902863 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:36.902863 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:36.902863 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:36.902863 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:36.902863 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:36.902863 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:08:36.902863 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:08:36.902863 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:08:36.902863 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:08:36.905314 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:36.905314 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:36.905314 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:08:36.905314 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:08:36.905314 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:08:36.905314 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:08:36.905314 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:36.905314 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:36.905314 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:36.905314 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:36.905314 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:36.905314 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:36.905314 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:36.905314 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:36.905314 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:36.905314 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:36.905314 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:36.905314 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:36.909365 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:36.909365 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:36.909365 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:08:36.909365 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:08:36.909365 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:08:36.909365 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:08:36.909365 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:08:36.909365 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:08:36.909365 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 25 19:08:36.909365 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:36.909365 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 256/496] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:08:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:36.909365 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42ca29-7310-486f-b67a-be4e81dc690d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:36.909365 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42ca29-7310-486f-b67a-be4e81dc690d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:36.911971 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42ca29-7310-486f-b67a-be4e81dc690d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:36.940955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8b42ca29-7310-486f-b67a-be4e81dc690d None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:36.942505 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8b42ca29-7310-486f-b67a-be4e81dc690d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:37.013462 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 241/497] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:08:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 19:08:37.015569 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668112bd-60d2-4645-87fd-99c3d62b6ccf None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:37.016084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668112bd-60d2-4645-87fd-99c3d62b6ccf None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:37.016580 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668112bd-60d2-4645-87fd-99c3d62b6ccf None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:37.019562 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 242/498] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:08:37.021177 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cf55e-b9b3-43de-a15d-4b0c544c6dc5 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:37.021624 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cf55e-b9b3-43de-a15d-4b0c544c6dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:37.022034 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cf55e-b9b3-43de-a15d-4b0c544c6dc5 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:37.027563 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 243/499] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:08:37.027563 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7cf55e-b9b3-43de-a15d-4b0c544c6dc5 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:37.066069 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 257/500] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:08:37.067401 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42ca29-7310-486f-b67a-be4e81dc690d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:37.088520 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:37.096466 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:37.186408 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7cf55e-b9b3-43de-a15d-4b0c544c6dc5 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:37.222015 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42ca29-7310-486f-b67a-be4e81dc690d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:37.232613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c20397-86c6-41f5-af47-60983a3a8b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:37.234327 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c20397-86c6-41f5-af47-60983a3a8b09 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:37.234795 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c20397-86c6-41f5-af47-60983a3a8b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:37.235301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c20397-86c6-41f5-af47-60983a3a8b09 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:37.236913 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c20397-86c6-41f5-af47-60983a3a8b09 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:37.237527 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c20397-86c6-41f5-af47-60983a3a8b09 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:37.240353 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c20397-86c6-41f5-af47-60983a3a8b09 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:37.270359 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d154e05-fe1c-4f51-b26f-6535054633d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:37.271362 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d154e05-fe1c-4f51-b26f-6535054633d9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:37.271730 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d154e05-fe1c-4f51-b26f-6535054633d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:37.272044 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d154e05-fe1c-4f51-b26f-6535054633d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:37.280007 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 244/501] 10.222.0.18 () {56 vars in 1137 bytes} [Mon Jul 25 19:08:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:37.280007 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753c828-ba3e-4728-bcde-51e65316f1cb None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:37.280007 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753c828-ba3e-4728-bcde-51e65316f1cb None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:37.280007 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753c828-ba3e-4728-bcde-51e65316f1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:37.287982 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d154e05-fe1c-4f51-b26f-6535054633d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:37.289808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d154e05-fe1c-4f51-b26f-6535054633d9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:37.308103 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6753c828-ba3e-4728-bcde-51e65316f1cb None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:37.310690 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6753c828-ba3e-4728-bcde-51e65316f1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:37.311577 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-9d154e05-fe1c-4f51-b26f-6535054633d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:37.311577 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:37.311577 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:37.311577 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:37.311577 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:37.311577 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:37.311577 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:37.311577 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:37.311577 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:37.311577 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:37.311577 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:37.311577 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:37.311577 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:08:37.311577 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:08:37.311577 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:08:37.313389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:08:37.313389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:37.313389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:37.313389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:37.313389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:37.313389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:37.313389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:37.313389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:37.313389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:37.313389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:37.313389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:37.313389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:37.313389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:37.313389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:37.313389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:37.313389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:37.313389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:37.315301 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:08:37.315301 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:08:37.315301 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:37.315301 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:37.315301 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 258/502] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:08:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:37.315301 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41071ba4-a131-4663-99fd-4dabb7f806f0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:37.315301 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41071ba4-a131-4663-99fd-4dabb7f806f0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:37.316064 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41071ba4-a131-4663-99fd-4dabb7f806f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:37.336397 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-41071ba4-a131-4663-99fd-4dabb7f806f0 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:37.337764 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-41071ba4-a131-4663-99fd-4dabb7f806f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:37.424192 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:37.439852 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 259/503] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:08:37.439852 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41071ba4-a131-4663-99fd-4dabb7f806f0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:37.467560 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 245/504] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:08:37.467560 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6753c828-ba3e-4728-bcde-51e65316f1cb None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:37.504717 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:37.600353 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41071ba4-a131-4663-99fd-4dabb7f806f0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:37.623229 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6753c828-ba3e-4728-bcde-51e65316f1cb None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:37.661853 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0234ad97-5d90-4dfa-aa56-d87bbc04b657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:37.662685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234ad97-5d90-4dfa-aa56-d87bbc04b657 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:37.662979 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234ad97-5d90-4dfa-aa56-d87bbc04b657 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:37.663268 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234ad97-5d90-4dfa-aa56-d87bbc04b657 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:37.667642 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e7253-ef19-4dd2-ae7c-d1baca08e2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:37.669281 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e7253-ef19-4dd2-ae7c-d1baca08e2c7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:37.670060 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e7253-ef19-4dd2-ae7c-d1baca08e2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:37.670572 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0e7253-ef19-4dd2-ae7c-d1baca08e2c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:37.679790 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0234ad97-5d90-4dfa-aa56-d87bbc04b657 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:08:37.689027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-0234ad97-5d90-4dfa-aa56-d87bbc04b657 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:08:37.689027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:37.689027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:37.689027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:37.689027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:37.689027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:37.689027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:37.689027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:37.689027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:37.689027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:37.689027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:37.689027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:37.689027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:08:37.689027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:08:37.689027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:08:37.689027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:08:37.690090 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:08:37.690090 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:08:37.690090 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:08:37.690090 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:08:37.690090 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:37.690090 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:37.690090 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:37.690090 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:37.690090 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:37.690090 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:37.690090 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:37.690090 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:37.690090 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:37.690090 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:37.690090 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:37.690090 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:37.690904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:37.690904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:37.690904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:37.690904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:37.690904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:08:37.690904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:08:37.690904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:08:37.690904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:08:37.690904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:08:37.690904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:08:37.690904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:08:37.690904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:37.690904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 260/505] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Jul 25 19:08:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:37.692074 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0234ad97-5d90-4dfa-aa56-d87bbc04b657 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:37.766115 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:37.803909 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0e7253-ef19-4dd2-ae7c-d1baca08e2c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:37.805641 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0e7253-ef19-4dd2-ae7c-d1baca08e2c7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:37.827852 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0234ad97-5d90-4dfa-aa56-d87bbc04b657 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:37.859123 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:37.894443 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec72d8-76fd-4c32-b238-fbee269a5125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:37.895566 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec72d8-76fd-4c32-b238-fbee269a5125 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:37.896005 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec72d8-76fd-4c32-b238-fbee269a5125 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:37.896385 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ec72d8-76fd-4c32-b238-fbee269a5125 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:37.897611 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ec72d8-76fd-4c32-b238-fbee269a5125 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:37.898041 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ec72d8-76fd-4c32-b238-fbee269a5125 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:37.900374 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ec72d8-76fd-4c32-b238-fbee269a5125 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:37.928519 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 261/506] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:08:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:37.928519 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec72d8-76fd-4c32-b238-fbee269a5125 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:37.987191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 246/507] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:08:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 19:08:37.988759 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e7253-ef19-4dd2-ae7c-d1baca08e2c7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:38.092836 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ec72d8-76fd-4c32-b238-fbee269a5125 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:38.120959 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:38.125327 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0e7253-ef19-4dd2-ae7c-d1baca08e2c7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:38.156313 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803543cc-edc1-4250-b817-580fc857452c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:38.157573 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803543cc-edc1-4250-b817-580fc857452c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:38.157948 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803543cc-edc1-4250-b817-580fc857452c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:38.158292 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803543cc-edc1-4250-b817-580fc857452c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:38.171796 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:38.174037 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803543cc-edc1-4250-b817-580fc857452c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:38.176320 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803543cc-edc1-4250-b817-580fc857452c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:38.180103 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035adb11-c1e0-4f78-8479-0964cc063474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:38.180645 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035adb11-c1e0-4f78-8479-0964cc063474 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:38.180989 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035adb11-c1e0-4f78-8479-0964cc063474 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:38.181338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035adb11-c1e0-4f78-8479-0964cc063474 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:38.195279 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035adb11-c1e0-4f78-8479-0964cc063474 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:08:38.197970 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-803543cc-edc1-4250-b817-580fc857452c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:38.197970 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:38.197970 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:38.197970 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:38.197970 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:38.197970 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:38.197970 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:38.197970 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:38.197970 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:38.197970 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:38.197970 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:38.197970 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:38.197970 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:08:38.197970 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:08:38.197970 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:08:38.200262 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:08:38.200262 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:38.200262 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:38.200262 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:38.200262 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:38.200262 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:38.200262 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:38.200262 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:38.200262 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:38.200262 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:38.200262 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:38.200262 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:38.200262 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:38.200262 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:38.200262 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:38.200262 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:38.200262 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:38.201973 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:08:38.201973 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:08:38.201973 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:38.201973 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:38.201973 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 262/508] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:08:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:38.208808 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-035adb11-c1e0-4f78-8479-0964cc063474 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:08:38.208808 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:38.208808 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:38.208808 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:38.208808 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:38.208808 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:38.208808 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:38.208808 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:38.208808 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:38.208808 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:38.208808 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:38.208808 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:38.208808 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:08:38.208808 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:08:38.208808 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:08:38.208808 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:08:38.212542 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:08:38.212542 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:08:38.212542 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:08:38.212542 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:08:38.212542 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:38.212542 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:38.212542 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:38.212542 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:38.212542 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:38.212542 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:38.212542 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:38.212542 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:38.212542 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:38.212542 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:38.212542 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:38.212542 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:38.214994 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:38.214994 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:38.214994 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:38.214994 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:38.214994 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:08:38.214994 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:08:38.214994 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:08:38.214994 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:08:38.214994 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:08:38.214994 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:08:38.214994 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:08:38.214994 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:38.217510 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803543cc-edc1-4250-b817-580fc857452c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:38.219101 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 247/509] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Jul 25 19:08:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:38.220164 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035adb11-c1e0-4f78-8479-0964cc063474 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:38.383213 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803543cc-edc1-4250-b817-580fc857452c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:38.392671 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035adb11-c1e0-4f78-8479-0964cc063474 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:38.440535 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e0dd9-defb-49bc-98a7-5ba2f240b34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:38.441547 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e0dd9-defb-49bc-98a7-5ba2f240b34c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:38.441997 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e0dd9-defb-49bc-98a7-5ba2f240b34c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:38.442392 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e0dd9-defb-49bc-98a7-5ba2f240b34c None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:38.443593 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9e0dd9-defb-49bc-98a7-5ba2f240b34c None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:38.444058 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9e0dd9-defb-49bc-98a7-5ba2f240b34c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:38.446365 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9e0dd9-defb-49bc-98a7-5ba2f240b34c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:38.456254 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc947b-33de-4bf6-b3ca-40c8437de53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:38.457596 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc947b-33de-4bf6-b3ca-40c8437de53f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:38.458067 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc947b-33de-4bf6-b3ca-40c8437de53f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:38.458522 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc947b-33de-4bf6-b3ca-40c8437de53f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:38.472140 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 263/510] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Jul 25 19:08:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:38.472401 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdc947b-33de-4bf6-b3ca-40c8437de53f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:38.473011 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e0dd9-defb-49bc-98a7-5ba2f240b34c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:38.474967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdc947b-33de-4bf6-b3ca-40c8437de53f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:38.505510 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-dcdc947b-33de-4bf6-b3ca-40c8437de53f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:08:38.505510 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:38.505510 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:38.505510 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:38.505510 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:38.505510 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:38.505510 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:38.505510 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:38.505510 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:38.505510 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:38.505510 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:38.505510 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:38.505510 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:08:38.505510 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:08:38.505510 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:08:38.506658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:08:38.506658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:38.506658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:38.506658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:38.506658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:38.506658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:38.506658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:38.506658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:38.506658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:38.506658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:38.506658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:38.506658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:38.506658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:38.506658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:38.506658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:38.506658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:38.506658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:38.507904 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:08:38.507904 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:08:38.507904 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:08:38.507904 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:08:38.507904 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:08:38.507904 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:08:38.507904 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:08:38.507904 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:38.508761 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 248/511] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:08:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:38.509953 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc947b-33de-4bf6-b3ca-40c8437de53f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:38.518175 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:38.589256 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:38.651767 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e0dd9-defb-49bc-98a7-5ba2f240b34c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:38.659364 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc947b-33de-4bf6-b3ca-40c8437de53f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:38.718293 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208ed34-99f6-4511-b03e-15ecd6b91626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:38.718924 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30447dd9-dfe1-445d-b1c8-9d1eb6f4befa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:38.719282 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208ed34-99f6-4511-b03e-15ecd6b91626 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:38.719707 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208ed34-99f6-4511-b03e-15ecd6b91626 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:38.719875 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30447dd9-dfe1-445d-b1c8-9d1eb6f4befa None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:38.720096 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30447dd9-dfe1-445d-b1c8-9d1eb6f4befa None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:38.720249 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208ed34-99f6-4511-b03e-15ecd6b91626 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:38.720396 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30447dd9-dfe1-445d-b1c8-9d1eb6f4befa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:38.721128 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d208ed34-99f6-4511-b03e-15ecd6b91626 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:38.721531 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d208ed34-99f6-4511-b03e-15ecd6b91626 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:38.723164 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d208ed34-99f6-4511-b03e-15ecd6b91626 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:38.734767 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30447dd9-dfe1-445d-b1c8-9d1eb6f4befa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:38.736597 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30447dd9-dfe1-445d-b1c8-9d1eb6f4befa None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:38.737984 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 249/512] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:38.738927 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208ed34-99f6-4511-b03e-15ecd6b91626 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:38.773782 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-30447dd9-dfe1-445d-b1c8-9d1eb6f4befa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:38.773782 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:38.773782 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:38.773782 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:38.773782 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:38.773782 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:38.773782 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:38.773782 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:38.773782 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:38.773782 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:38.773782 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:38.773782 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:38.773782 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:08:38.773782 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:08:38.773782 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:08:38.774887 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:08:38.774887 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:38.774887 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:38.774887 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:38.774887 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:38.774887 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:38.774887 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:38.774887 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:38.774887 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:38.774887 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:38.774887 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:38.774887 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:38.774887 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:38.774887 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:38.774887 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:38.774887 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:38.774887 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:38.776383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:08:38.776383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:08:38.776383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:38.776383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:38.776383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 264/513] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:08:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:38.777400 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30447dd9-dfe1-445d-b1c8-9d1eb6f4befa None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:38.879213 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208ed34-99f6-4511-b03e-15ecd6b91626 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:38.892233 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:38.923454 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:38.924426 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269cea4c-0d06-4fbc-b622-25b8152034d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:38.925338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269cea4c-0d06-4fbc-b622-25b8152034d0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:38.925808 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269cea4c-0d06-4fbc-b622-25b8152034d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:38.926536 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269cea4c-0d06-4fbc-b622-25b8152034d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:38.931536 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269cea4c-0d06-4fbc-b622-25b8152034d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:38.932065 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269cea4c-0d06-4fbc-b622-25b8152034d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:38.933706 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269cea4c-0d06-4fbc-b622-25b8152034d0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:38.949838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 250/514] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Jul 25 19:08:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:38.951319 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269cea4c-0d06-4fbc-b622-25b8152034d0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:38.962946 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30447dd9-dfe1-445d-b1c8-9d1eb6f4befa None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:39.032729 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797064de-d112-4555-8368-b5e4d2047d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:39.034163 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797064de-d112-4555-8368-b5e4d2047d8e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:39.034643 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797064de-d112-4555-8368-b5e4d2047d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:39.035139 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797064de-d112-4555-8368-b5e4d2047d8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:39.056615 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797064de-d112-4555-8368-b5e4d2047d8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:39.060161 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797064de-d112-4555-8368-b5e4d2047d8e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:39.082974 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269cea4c-0d06-4fbc-b622-25b8152034d0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:39.097572 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-797064de-d112-4555-8368-b5e4d2047d8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:39.097572 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:39.097572 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:39.097572 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:39.097572 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:39.097572 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:39.097572 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:39.097572 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:39.097572 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:39.097572 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:39.097572 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:39.097572 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:39.097572 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:08:39.097572 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:08:39.097572 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:08:39.098992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:08:39.098992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:39.098992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:39.098992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:39.098992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:39.098992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:39.098992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:39.098992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:39.098992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:39.098992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:39.098992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:39.098992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:39.098992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:39.098992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:39.098992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:39.098992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:39.098992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:39.100092 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:08:39.100092 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:08:39.100092 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:39.100092 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:39.100092 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 265/515] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:08:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:39.101363 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797064de-d112-4555-8368-b5e4d2047d8e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:39.132937 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9b51a5-8765-43e4-b89b-9d2a6d4bc418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:39.133616 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b51a5-8765-43e4-b89b-9d2a6d4bc418 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:39.133908 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b51a5-8765-43e4-b89b-9d2a6d4bc418 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:39.134182 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b51a5-8765-43e4-b89b-9d2a6d4bc418 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:39.134997 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9b51a5-8765-43e4-b89b-9d2a6d4bc418 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:39.135314 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9b51a5-8765-43e4-b89b-9d2a6d4bc418 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:39.137066 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9b51a5-8765-43e4-b89b-9d2a6d4bc418 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:39.159781 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 251/516] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Jul 25 19:08:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:39.160717 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9b51a5-8765-43e4-b89b-9d2a6d4bc418 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:39.205396 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:39.278861 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797064de-d112-4555-8368-b5e4d2047d8e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:39.317581 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:39.320859 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9b51a5-8765-43e4-b89b-9d2a6d4bc418 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:39.324339 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd542253-05f2-43af-80d2-985ba2279110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:39.325706 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd542253-05f2-43af-80d2-985ba2279110 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:39.326075 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd542253-05f2-43af-80d2-985ba2279110 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:39.326479 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd542253-05f2-43af-80d2-985ba2279110 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:39.327271 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd542253-05f2-43af-80d2-985ba2279110 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:39.329062 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd542253-05f2-43af-80d2-985ba2279110 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:39.366598 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 266/517] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Jul 25 19:08:39 2022] POST /identity/v3/users => generated 247 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:08:39.368293 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd542253-05f2-43af-80d2-985ba2279110 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:39.380225 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cd9d38-e0b2-4c49-b33e-f7102d1f1218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:39.381355 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cd9d38-e0b2-4c49-b33e-f7102d1f1218 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:39.381799 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cd9d38-e0b2-4c49-b33e-f7102d1f1218 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:39.382360 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cd9d38-e0b2-4c49-b33e-f7102d1f1218 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:39.383461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cd9d38-e0b2-4c49-b33e-f7102d1f1218 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:39.383988 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cd9d38-e0b2-4c49-b33e-f7102d1f1218 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:39.386402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cd9d38-e0b2-4c49-b33e-f7102d1f1218 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:39.409829 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 252/518] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:08:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:39.411683 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cd9d38-e0b2-4c49-b33e-f7102d1f1218 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:39.558594 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd542253-05f2-43af-80d2-985ba2279110 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:39.614103 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:39.629011 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cd9d38-e0b2-4c49-b33e-f7102d1f1218 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:39.630053 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f724e628-5b26-4825-9d78-3b75591996c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:39.631248 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f724e628-5b26-4825-9d78-3b75591996c8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:39.631752 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f724e628-5b26-4825-9d78-3b75591996c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:39.632381 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f724e628-5b26-4825-9d78-3b75591996c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:39.654808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f724e628-5b26-4825-9d78-3b75591996c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:39.656459 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f724e628-5b26-4825-9d78-3b75591996c8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:39.690071 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-f724e628-5b26-4825-9d78-3b75591996c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:39.690071 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:39.690071 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:39.690071 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:39.690071 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:39.690071 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:39.690071 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:39.690071 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:39.690071 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:39.690071 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:39.690071 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:39.690071 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:39.690071 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:08:39.690071 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:08:39.690071 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:08:39.691985 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:08:39.691985 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:39.691985 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:39.691985 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:39.691985 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:39.691985 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:39.691985 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:39.691985 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:39.691985 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:39.691985 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:39.691985 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:39.691985 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:39.691985 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:39.691985 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:39.691985 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:39.691985 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:39.691985 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:39.693926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:08:39.693926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:08:39.693926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:39.693926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:39.693926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 267/519] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:08:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:39.693926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f724e628-5b26-4825-9d78-3b75591996c8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:39.711306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b352c347-5a59-4c72-ae36-87a53a0bfefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:39.712239 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352c347-5a59-4c72-ae36-87a53a0bfefc None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:39.712620 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352c347-5a59-4c72-ae36-87a53a0bfefc None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:39.713110 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352c347-5a59-4c72-ae36-87a53a0bfefc None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:39.714512 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b352c347-5a59-4c72-ae36-87a53a0bfefc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:39.714833 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b352c347-5a59-4c72-ae36-87a53a0bfefc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:39.717682 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b352c347-5a59-4c72-ae36-87a53a0bfefc None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:39.736237 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 253/520] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:08:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:39.738108 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b352c347-5a59-4c72-ae36-87a53a0bfefc None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:39.774221 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:39.865301 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f724e628-5b26-4825-9d78-3b75591996c8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:39.913172 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c1f78e-16a4-46c5-a768-5e177bf662bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:39.914073 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c1f78e-16a4-46c5-a768-5e177bf662bf None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:39.914501 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c1f78e-16a4-46c5-a768-5e177bf662bf None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:39.914930 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c1f78e-16a4-46c5-a768-5e177bf662bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:39.916171 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c1f78e-16a4-46c5-a768-5e177bf662bf None admin] RBAC: Adding query filter params (role.id=23718d83b0864b16b0071e8ffe0b5861, scope.project.id=191069caccd04233b012585e29cfa18b, user.id=860e7d26ad364795bb25836aadd4ac7b) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:39.916627 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c1f78e-16a4-46c5-a768-5e177bf662bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:39.918975 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c1f78e-16a4-46c5-a768-5e177bf662bf None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:39.928136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 268/521] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:08:39 2022] GET /identity/v3/role_assignments?user.id=860e7d26ad364795bb25836aadd4ac7b&scope.project.id=191069caccd04233b012585e29cfa18b&role.id=23718d83b0864b16b0071e8ffe0b5861 => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:39.929651 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c1f78e-16a4-46c5-a768-5e177bf662bf None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:39.965768 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b352c347-5a59-4c72-ae36-87a53a0bfefc None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:40.054950 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6458ac0e-ba28-42c0-bc9a-9f8ce4e81b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:40.056379 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458ac0e-ba28-42c0-bc9a-9f8ce4e81b64 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:40.056845 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458ac0e-ba28-42c0-bc9a-9f8ce4e81b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:40.057311 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458ac0e-ba28-42c0-bc9a-9f8ce4e81b64 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:40.070123 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6458ac0e-ba28-42c0-bc9a-9f8ce4e81b64 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:40.072878 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6458ac0e-ba28-42c0-bc9a-9f8ce4e81b64 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:40.100110 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-6458ac0e-ba28-42c0-bc9a-9f8ce4e81b64 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:08:40.100110 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:40.100110 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:40.100110 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:40.100110 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:40.100110 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:40.100110 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:40.100110 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:40.100110 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:40.100110 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:40.100110 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:40.100110 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:40.100110 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:08:40.100110 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:08:40.100110 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:08:40.101754 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:08:40.101754 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:40.101754 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:40.101754 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:40.101754 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:40.101754 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:40.101754 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:40.101754 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:40.101754 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:40.101754 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:40.101754 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:40.101754 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:40.101754 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:40.101754 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:40.101754 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:40.101754 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:40.101754 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:40.104900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:08:40.104900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:08:40.104900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:08:40.104900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:08:40.104900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:08:40.104900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:08:40.104900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:08:40.104900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:40.104900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 254/522] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:08:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:40.104900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6458ac0e-ba28-42c0-bc9a-9f8ce4e81b64 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:40.140185 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:40.156556 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c1f78e-16a4-46c5-a768-5e177bf662bf None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:40.161670 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:40.236787 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902a9870-0564-4214-8725-ed858d44fc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:40.237873 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a9870-0564-4214-8725-ed858d44fc45 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:40.238551 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a9870-0564-4214-8725-ed858d44fc45 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:40.239002 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a9870-0564-4214-8725-ed858d44fc45 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:40.240533 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902a9870-0564-4214-8725-ed858d44fc45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:40.241143 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902a9870-0564-4214-8725-ed858d44fc45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:40.243884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902a9870-0564-4214-8725-ed858d44fc45 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:40.282629 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 269/523] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:08:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:40.284203 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902a9870-0564-4214-8725-ed858d44fc45 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:40.334002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6458ac0e-ba28-42c0-bc9a-9f8ce4e81b64 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:40.390732 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2bb9b0-21f7-4117-a0a5-300642c794ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:40.391624 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2bb9b0-21f7-4117-a0a5-300642c794ba None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:40.391956 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2bb9b0-21f7-4117-a0a5-300642c794ba None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:40.392458 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2bb9b0-21f7-4117-a0a5-300642c794ba None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:40.403560 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2bb9b0-21f7-4117-a0a5-300642c794ba None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:40.405297 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2bb9b0-21f7-4117-a0a5-300642c794ba None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:40.424900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-ee2bb9b0-21f7-4117-a0a5-300642c794ba None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 19:08:40.424900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:40.424900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:40.424900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:40.424900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:40.424900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:40.424900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:40.424900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:40.424900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:40.424900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:40.424900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:40.424900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:40.424900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:08:40.424900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:08:40.424900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:08:40.424900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:08:40.426647 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:40.426647 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:40.426647 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:08:40.426647 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:08:40.426647 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:08:40.426647 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:08:40.426647 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:40.426647 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:40.426647 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:40.426647 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:40.426647 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:40.426647 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:40.426647 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:40.426647 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:40.426647 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:40.426647 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:40.426647 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:40.426647 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:40.429929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:40.429929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:40.429929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:08:40.429929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:08:40.429929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:08:40.429929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:08:40.429929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:08:40.429929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:08:40.429929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 19:08:40.429929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:40.429929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 255/524] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:08:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:40.429929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2bb9b0-21f7-4117-a0a5-300642c794ba None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:40.503479 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902a9870-0564-4214-8725-ed858d44fc45 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:40.564992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f824128-d9d4-4721-ac1e-ba773368e896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:40.565906 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f824128-d9d4-4721-ac1e-ba773368e896 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:40.566292 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f824128-d9d4-4721-ac1e-ba773368e896 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:40.566668 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f824128-d9d4-4721-ac1e-ba773368e896 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:40.567728 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f824128-d9d4-4721-ac1e-ba773368e896 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:40.568146 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f824128-d9d4-4721-ac1e-ba773368e896 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:40.569744 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f824128-d9d4-4721-ac1e-ba773368e896 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:40.571569 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:40.597213 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 270/525] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Jul 25 19:08:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:40.598614 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f824128-d9d4-4721-ac1e-ba773368e896 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:40.639646 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2bb9b0-21f7-4117-a0a5-300642c794ba None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:40.644735 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:40.729531 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54959c86-c44c-4689-87fa-409221e66add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:40.732207 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54959c86-c44c-4689-87fa-409221e66add None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:40.733278 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54959c86-c44c-4689-87fa-409221e66add None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:40.734448 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54959c86-c44c-4689-87fa-409221e66add None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:40.749271 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54959c86-c44c-4689-87fa-409221e66add None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:40.751035 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54959c86-c44c-4689-87fa-409221e66add None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:40.766656 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f824128-d9d4-4721-ac1e-ba773368e896 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:40.768832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-54959c86-c44c-4689-87fa-409221e66add None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 19:08:40.768832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:40.768832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:40.768832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:40.768832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:40.768832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:40.768832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:40.768832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:40.768832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:40.768832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:40.768832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:40.768832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:40.768832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:08:40.768832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:08:40.768832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:08:40.768832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:08:40.770838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:40.770838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:40.770838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:08:40.770838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:08:40.770838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:08:40.770838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:08:40.770838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:40.770838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:40.770838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:40.770838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:40.770838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:40.770838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:40.770838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:40.770838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:40.770838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:40.770838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:40.770838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:40.770838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:40.773017 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:40.773017 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:40.773017 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:08:40.773017 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:08:40.773017 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:08:40.773017 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:08:40.773017 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:08:40.773017 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:08:40.773017 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 19:08:40.773017 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:40.774084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 256/526] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Jul 25 19:08:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:40.779841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54959c86-c44c-4689-87fa-409221e66add None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:40.829158 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5927dd-ae4d-4148-922f-e233d921b6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:40.829947 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5927dd-ae4d-4148-922f-e233d921b6b1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:40.830243 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5927dd-ae4d-4148-922f-e233d921b6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:40.830523 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5927dd-ae4d-4148-922f-e233d921b6b1 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:40.831399 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5927dd-ae4d-4148-922f-e233d921b6b1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:40.831768 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5927dd-ae4d-4148-922f-e233d921b6b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:40.833269 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5927dd-ae4d-4148-922f-e233d921b6b1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:40.880480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 271/527] 10.222.0.18 () {56 vars in 1137 bytes} [Mon Jul 25 19:08:40 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:40.881461 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5927dd-ae4d-4148-922f-e233d921b6b1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:40.919143 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54959c86-c44c-4689-87fa-409221e66add None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:40.960321 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:40.968985 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7562cf-e686-43c8-b1dd-ce6b7605cd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:40.970023 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7562cf-e686-43c8-b1dd-ce6b7605cd33 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:40.970023 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7562cf-e686-43c8-b1dd-ce6b7605cd33 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:40.970280 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7562cf-e686-43c8-b1dd-ce6b7605cd33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:40.971470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7562cf-e686-43c8-b1dd-ce6b7605cd33 None admin] RBAC: Adding query filter params (role.id=23718d83b0864b16b0071e8ffe0b5861, scope.project.id=191069caccd04233b012585e29cfa18b, user.id=98560723b49940d38a6a5e9cf3f5d3b4) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:40.971827 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7562cf-e686-43c8-b1dd-ce6b7605cd33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:40.973361 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7562cf-e686-43c8-b1dd-ce6b7605cd33 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:40.981785 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 257/528] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:08:40 2022] GET /identity/v3/role_assignments?user.id=98560723b49940d38a6a5e9cf3f5d3b4&scope.project.id=191069caccd04233b012585e29cfa18b&role.id=23718d83b0864b16b0071e8ffe0b5861 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:40.982813 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7562cf-e686-43c8-b1dd-ce6b7605cd33 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:40.996415 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:41.040461 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5927dd-ae4d-4148-922f-e233d921b6b1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:41.116065 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e659e925-2bbf-4d03-8074-870225ef1ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:41.116771 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659e925-2bbf-4d03-8074-870225ef1ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:41.117059 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659e925-2bbf-4d03-8074-870225ef1ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:41.117328 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659e925-2bbf-4d03-8074-870225ef1ec4 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:41.118195 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e659e925-2bbf-4d03-8074-870225ef1ec4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:41.118506 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e659e925-2bbf-4d03-8074-870225ef1ec4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:41.120230 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e659e925-2bbf-4d03-8074-870225ef1ec4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:41.135122 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7562cf-e686-43c8-b1dd-ce6b7605cd33 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:41.183750 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 272/529] 10.222.0.18 () {56 vars in 1139 bytes} [Mon Jul 25 19:08:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:41.196858 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e659e925-2bbf-4d03-8074-870225ef1ec4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:41.203546 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40421e54-d882-43a6-b947-fa080285ce8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:41.203546 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40421e54-d882-43a6-b947-fa080285ce8f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:41.203546 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40421e54-d882-43a6-b947-fa080285ce8f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:41.203546 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40421e54-d882-43a6-b947-fa080285ce8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:41.349969 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:41.363096 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40421e54-d882-43a6-b947-fa080285ce8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:41.364847 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40421e54-d882-43a6-b947-fa080285ce8f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:41.506735 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 258/530] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:08:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 19:08:41.516015 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40421e54-d882-43a6-b947-fa080285ce8f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:41.554019 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e659e925-2bbf-4d03-8074-870225ef1ec4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:41.623478 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4c216-adb7-4456-a784-70e85416bfe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:41.624277 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4c216-adb7-4456-a784-70e85416bfe3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:41.624571 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4c216-adb7-4456-a784-70e85416bfe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:41.624860 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4c216-adb7-4456-a784-70e85416bfe3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:41.644169 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:41.696725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:41.713556 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40421e54-d882-43a6-b947-fa080285ce8f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:41.780604 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd4c216-adb7-4456-a784-70e85416bfe3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:41.782144 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd4c216-adb7-4456-a784-70e85416bfe3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:41.803340 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe1dfb-a012-400e-bbb1-c9e9fdfe5f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:41.804234 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe1dfb-a012-400e-bbb1-c9e9fdfe5f6c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:41.804526 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe1dfb-a012-400e-bbb1-c9e9fdfe5f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:41.804809 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe1dfb-a012-400e-bbb1-c9e9fdfe5f6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:41.828310 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbe1dfb-a012-400e-bbb1-c9e9fdfe5f6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:41.829876 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbe1dfb-a012-400e-bbb1-c9e9fdfe5f6c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:41.868065 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-5dbe1dfb-a012-400e-bbb1-c9e9fdfe5f6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:41.868065 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:41.868065 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:41.868065 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:41.868065 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:41.868065 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:41.868065 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:41.868065 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:41.868065 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:41.868065 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:41.868065 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:41.868065 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:41.868065 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:08:41.868065 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:08:41.868065 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:08:41.869189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:08:41.869189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:41.869189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:41.869189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:41.869189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:41.869189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:41.869189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:41.869189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:41.869189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:41.869189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:41.869189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:41.869189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:41.869189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:41.869189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:41.869189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:41.869189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:41.869189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:41.870080 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:08:41.870080 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:08:41.870080 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:41.870080 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:41.870080 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 259/531] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:08:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:41.884765 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe1dfb-a012-400e-bbb1-c9e9fdfe5f6c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:42.051670 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 273/532] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:08:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 856 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 19:08:42.065299 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4c216-adb7-4456-a784-70e85416bfe3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:42.177747 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:42.187553 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe1dfb-a012-400e-bbb1-c9e9fdfe5f6c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:42.258052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a6becc-0d42-4bef-aae5-45a758d0404a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:42.264645 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6becc-0d42-4bef-aae5-45a758d0404a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:42.268471 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6becc-0d42-4bef-aae5-45a758d0404a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:42.269027 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6becc-0d42-4bef-aae5-45a758d0404a None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:42.270504 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a6becc-0d42-4bef-aae5-45a758d0404a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:42.271073 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a6becc-0d42-4bef-aae5-45a758d0404a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:42.273960 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a6becc-0d42-4bef-aae5-45a758d0404a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:42.309979 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 260/533] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:08:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:42.311049 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6428726-bccf-4251-b43b-0362c02c6c69 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:42.311602 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6428726-bccf-4251-b43b-0362c02c6c69 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:42.312119 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6428726-bccf-4251-b43b-0362c02c6c69 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:42.315294 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 261/534] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:08:42.316891 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6428726-bccf-4251-b43b-0362c02c6c69 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:42.334297 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:42.349458 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4c216-adb7-4456-a784-70e85416bfe3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:42.424840 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3d7b30-d1e5-439e-81bc-2f03ed5d0874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:42.426148 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d7b30-d1e5-439e-81bc-2f03ed5d0874 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:42.427458 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d7b30-d1e5-439e-81bc-2f03ed5d0874 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:42.429105 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d7b30-d1e5-439e-81bc-2f03ed5d0874 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:42.447772 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3d7b30-d1e5-439e-81bc-2f03ed5d0874 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:42.451156 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3d7b30-d1e5-439e-81bc-2f03ed5d0874 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:42.478627 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-4d3d7b30-d1e5-439e-81bc-2f03ed5d0874 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:42.478627 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:42.478627 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:42.478627 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:42.478627 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:42.478627 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:42.478627 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:42.478627 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:42.478627 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:42.478627 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:42.478627 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:42.478627 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:42.478627 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:08:42.478627 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:08:42.478627 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:08:42.479608 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:08:42.479608 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:42.479608 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:42.479608 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:42.479608 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:42.479608 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:42.479608 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:42.479608 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:42.479608 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:42.479608 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:42.479608 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:42.479608 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:42.479608 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:42.479608 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:42.479608 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:42.479608 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:42.479608 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:42.480482 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:08:42.480482 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:08:42.480482 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:42.480482 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:42.483266 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 274/535] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:08:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 419 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:42.484945 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd1219-bc93-4b6c-b690-7e0e4048b12a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:42.485387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd1219-bc93-4b6c-b690-7e0e4048b12a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:42.485844 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd1219-bc93-4b6c-b690-7e0e4048b12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:42.514275 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-45bd1219-bc93-4b6c-b690-7e0e4048b12a None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:42.516208 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-45bd1219-bc93-4b6c-b690-7e0e4048b12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:42.539840 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6428726-bccf-4251-b43b-0362c02c6c69 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:42.604940 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999e2db-c06c-4485-abf2-d0940e8f7a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:42.606680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999e2db-c06c-4485-abf2-d0940e8f7a74 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:42.607143 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999e2db-c06c-4485-abf2-d0940e8f7a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:42.607606 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999e2db-c06c-4485-abf2-d0940e8f7a74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:42.619265 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9999e2db-c06c-4485-abf2-d0940e8f7a74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:42.623452 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9999e2db-c06c-4485-abf2-d0940e8f7a74 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:42.627631 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 275/536] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:08:42.629073 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bd1219-bc93-4b6c-b690-7e0e4048b12a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:42.644658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-9999e2db-c06c-4485-abf2-d0940e8f7a74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:08:42.644658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:42.644658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:42.644658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:42.644658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:42.644658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:42.644658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:42.644658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:42.644658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:42.644658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:42.644658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:42.644658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:42.644658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:08:42.644658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:08:42.644658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:08:42.645626 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:08:42.645626 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:42.645626 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:42.645626 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:42.645626 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:42.645626 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:42.645626 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:42.645626 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:42.645626 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:42.645626 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:42.645626 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:42.645626 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:42.645626 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:42.645626 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:42.645626 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:42.645626 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:42.645626 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:42.646597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:08:42.646597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:08:42.646597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:08:42.646597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:08:42.646597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:08:42.646597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:08:42.646597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:08:42.646597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:42.651918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 262/537] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:08:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:42.654231 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999e2db-c06c-4485-abf2-d0940e8f7a74 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:42.678224 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:42.786727 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:42.837943 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bd1219-bc93-4b6c-b690-7e0e4048b12a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:42.841486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999e2db-c06c-4485-abf2-d0940e8f7a74 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:42.897916 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef580629-2bb6-4687-ae56-3397e0da5d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:42.899311 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef580629-2bb6-4687-ae56-3397e0da5d28 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:42.900078 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef580629-2bb6-4687-ae56-3397e0da5d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:42.900588 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef580629-2bb6-4687-ae56-3397e0da5d28 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:42.901799 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef580629-2bb6-4687-ae56-3397e0da5d28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:42.902403 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef580629-2bb6-4687-ae56-3397e0da5d28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:42.905174 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef580629-2bb6-4687-ae56-3397e0da5d28 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:42.921298 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4645c268-e423-4ec5-a9ad-43364a54e4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:42.921298 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645c268-e423-4ec5-a9ad-43364a54e4e5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:42.921298 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645c268-e423-4ec5-a9ad-43364a54e4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:42.921298 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645c268-e423-4ec5-a9ad-43364a54e4e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:42.922677 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4645c268-e423-4ec5-a9ad-43364a54e4e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:42.923236 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4645c268-e423-4ec5-a9ad-43364a54e4e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:42.926020 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4645c268-e423-4ec5-a9ad-43364a54e4e5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:42.930113 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 263/538] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Jul 25 19:08:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:08:42.932231 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef580629-2bb6-4687-ae56-3397e0da5d28 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:42.953797 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 276/539] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:08:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:42.955309 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4645c268-e423-4ec5-a9ad-43364a54e4e5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:43.081790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef580629-2bb6-4687-ae56-3397e0da5d28 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:43.132872 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:43.138508 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:43.167873 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95fc57b-9b52-42d1-a6a9-e1250a32b78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:43.167873 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95fc57b-9b52-42d1-a6a9-e1250a32b78d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:43.167873 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95fc57b-9b52-42d1-a6a9-e1250a32b78d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:43.167873 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95fc57b-9b52-42d1-a6a9-e1250a32b78d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:43.182430 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4645c268-e423-4ec5-a9ad-43364a54e4e5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:43.208067 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95fc57b-9b52-42d1-a6a9-e1250a32b78d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:08:43.233008 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b95fc57b-9b52-42d1-a6a9-e1250a32b78d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:08:43.233008 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:43.233008 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:43.233008 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:43.233008 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:43.233008 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:43.233008 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:43.233008 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:43.233008 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:43.233008 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:43.233008 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:43.233008 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:43.233008 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:08:43.233008 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:08:43.233008 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:08:43.233008 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:08:43.235091 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:08:43.235091 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:08:43.235091 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:08:43.235091 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:08:43.235091 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:43.235091 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:43.235091 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:43.235091 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:43.235091 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:43.235091 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:43.235091 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:43.235091 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:43.235091 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:43.235091 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:43.235091 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:43.235091 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:43.236843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:43.236843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:43.236843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:43.236843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:43.236843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:08:43.236843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:08:43.236843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:08:43.236843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:08:43.236843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:08:43.236843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:08:43.236843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:08:43.236843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:43.242541 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 264/540] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Jul 25 19:08:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:43.245461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95fc57b-9b52-42d1-a6a9-e1250a32b78d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:43.257149 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010c36b-9acf-47f9-a7f8-0147d49f5839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:43.260321 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010c36b-9acf-47f9-a7f8-0147d49f5839 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:43.261294 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010c36b-9acf-47f9-a7f8-0147d49f5839 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:43.262154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010c36b-9acf-47f9-a7f8-0147d49f5839 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:43.264243 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e010c36b-9acf-47f9-a7f8-0147d49f5839 None admin] RBAC: Adding query filter params (role.id=23718d83b0864b16b0071e8ffe0b5861, scope.project.id=191069caccd04233b012585e29cfa18b, user.id=2aa720a35b9844848abcc42a4c4d6a2d) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:43.265194 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e010c36b-9acf-47f9-a7f8-0147d49f5839 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:43.270211 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e010c36b-9acf-47f9-a7f8-0147d49f5839 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:43.287871 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 277/541] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:08:42 2022] GET /identity/v3/role_assignments?user.id=2aa720a35b9844848abcc42a4c4d6a2d&scope.project.id=191069caccd04233b012585e29cfa18b&role.id=23718d83b0864b16b0071e8ffe0b5861 => generated 259 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:43.289988 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010c36b-9acf-47f9-a7f8-0147d49f5839 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:43.464640 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95fc57b-9b52-42d1-a6a9-e1250a32b78d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:43.521357 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2988d-a544-441f-9ce4-3a40d813068c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:43.521357 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2988d-a544-441f-9ce4-3a40d813068c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:43.521735 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2988d-a544-441f-9ce4-3a40d813068c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:43.521893 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2988d-a544-441f-9ce4-3a40d813068c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:43.531259 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c2988d-a544-441f-9ce4-3a40d813068c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:43.533427 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c2988d-a544-441f-9ce4-3a40d813068c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:43.546685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010c36b-9acf-47f9-a7f8-0147d49f5839 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:43.552035 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b2c2988d-a544-441f-9ce4-3a40d813068c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:08:43.552035 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:43.552035 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:43.552035 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:43.552035 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:43.552035 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:43.552035 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:43.552035 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:43.552035 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:43.552035 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:43.552035 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:43.552035 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:43.552035 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:08:43.552035 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:08:43.552035 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:08:43.553663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:08:43.553663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:43.553663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:43.553663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:43.553663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:43.553663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:43.553663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:43.553663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:43.553663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:43.553663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:43.553663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:43.553663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:43.553663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:43.553663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:43.553663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:43.553663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:43.553663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:43.555302 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:08:43.555302 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:08:43.555302 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:08:43.555302 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:08:43.555302 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:08:43.555302 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:08:43.555302 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:08:43.555302 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:43.561137 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 265/542] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:08:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:43.563309 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7ba51-9708-42d5-901b-82e65c31b962 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:43.563309 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7ba51-9708-42d5-901b-82e65c31b962 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:43.563309 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7ba51-9708-42d5-901b-82e65c31b962 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:43.564577 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 266/543] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:08:43.570699 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a7ba51-9708-42d5-901b-82e65c31b962 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:43.610103 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48c7f9-a331-4dd3-ac8c-43d1bf57a319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:43.611095 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48c7f9-a331-4dd3-ac8c-43d1bf57a319 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:43.611494 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48c7f9-a331-4dd3-ac8c-43d1bf57a319 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:43.612051 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48c7f9-a331-4dd3-ac8c-43d1bf57a319 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:43.612964 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f48c7f9-a331-4dd3-ac8c-43d1bf57a319 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:43.613415 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f48c7f9-a331-4dd3-ac8c-43d1bf57a319 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:43.615595 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f48c7f9-a331-4dd3-ac8c-43d1bf57a319 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:43.627667 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 278/544] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Jul 25 19:08:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:43.629518 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736ef31-65f2-41ce-8764-e9308ed61632 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:43.629899 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736ef31-65f2-41ce-8764-e9308ed61632 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:43.630260 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736ef31-65f2-41ce-8764-e9308ed61632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:43.639880 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:43.658820 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2736ef31-65f2-41ce-8764-e9308ed61632 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:43.661430 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2736ef31-65f2-41ce-8764-e9308ed61632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:43.666125 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:43.719558 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a7ba51-9708-42d5-901b-82e65c31b962 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:43.774133 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50655ac4-0991-4fd1-ab33-5dc671026d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:43.776680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50655ac4-0991-4fd1-ab33-5dc671026d5b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:43.776989 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50655ac4-0991-4fd1-ab33-5dc671026d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:43.777292 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50655ac4-0991-4fd1-ab33-5dc671026d5b None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:43.782530 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50655ac4-0991-4fd1-ab33-5dc671026d5b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:43.782530 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50655ac4-0991-4fd1-ab33-5dc671026d5b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:43.782530 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50655ac4-0991-4fd1-ab33-5dc671026d5b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:43.803281 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 267/545] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Jul 25 19:08:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:43.804091 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50655ac4-0991-4fd1-ab33-5dc671026d5b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:43.812678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 279/546] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:08:43.814340 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2736ef31-65f2-41ce-8764-e9308ed61632 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:43.960157 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50655ac4-0991-4fd1-ab33-5dc671026d5b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:43.973223 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:44.006225 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e68abc-0725-4ca4-92b1-1f6311e80797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:44.007236 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e68abc-0725-4ca4-92b1-1f6311e80797 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:44.007628 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e68abc-0725-4ca4-92b1-1f6311e80797 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:44.007980 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e68abc-0725-4ca4-92b1-1f6311e80797 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:44.022316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e68abc-0725-4ca4-92b1-1f6311e80797 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:44.024134 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e68abc-0725-4ca4-92b1-1f6311e80797 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:44.034953 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2736ef31-65f2-41ce-8764-e9308ed61632 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:44.046057 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b2e68abc-0725-4ca4-92b1-1f6311e80797 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:44.046057 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:44.046057 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:44.046057 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:44.046057 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:44.046057 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:44.046057 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:44.046057 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:44.046057 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:44.046057 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:44.046057 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:44.046057 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:44.046057 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:08:44.046057 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:08:44.046057 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:08:44.047014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:08:44.047014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:44.047014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:44.047014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:44.047014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:44.047014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:44.047014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:44.047014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:44.047014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:44.047014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:44.047014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:44.047014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:44.047014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:44.047014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:44.047014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:44.047014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:44.047014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:44.048571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:08:44.048571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:08:44.048571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:44.048571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:44.048571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 268/547] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:08:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:44.048571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e68abc-0725-4ca4-92b1-1f6311e80797 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:44.115451 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db0b1e-4abd-412d-aed4-65933bc8e2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:44.116852 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db0b1e-4abd-412d-aed4-65933bc8e2c8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:44.117351 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db0b1e-4abd-412d-aed4-65933bc8e2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:44.117827 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db0b1e-4abd-412d-aed4-65933bc8e2c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:44.119747 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12db0b1e-4abd-412d-aed4-65933bc8e2c8 None admin] RBAC: Adding query filter params (role.id=23718d83b0864b16b0071e8ffe0b5861, scope.project.id=191069caccd04233b012585e29cfa18b, user.id=181c84715c094100b414cea3c4919eb7) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:44.120291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12db0b1e-4abd-412d-aed4-65933bc8e2c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:44.123870 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12db0b1e-4abd-412d-aed4-65933bc8e2c8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:44.135866 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 280/548] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:08:43 2022] GET /identity/v3/role_assignments?user.id=181c84715c094100b414cea3c4919eb7&scope.project.id=191069caccd04233b012585e29cfa18b&role.id=23718d83b0864b16b0071e8ffe0b5861 => generated 259 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:44.137367 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db0b1e-4abd-412d-aed4-65933bc8e2c8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:44.142520 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:44.182878 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e68abc-0725-4ca4-92b1-1f6311e80797 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:44.248921 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d3af05-dc50-40df-bce6-e6313e7e8f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:44.250392 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3af05-dc50-40df-bce6-e6313e7e8f29 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:44.256197 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3af05-dc50-40df-bce6-e6313e7e8f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:44.256751 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3af05-dc50-40df-bce6-e6313e7e8f29 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:44.300625 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d3af05-dc50-40df-bce6-e6313e7e8f29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:08:44.309418 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-14d3af05-dc50-40df-bce6-e6313e7e8f29 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:08:44.309418 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:44.309418 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:44.309418 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:44.309418 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:44.309418 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:44.309418 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:44.309418 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:44.309418 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:44.309418 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:44.309418 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:44.309418 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:44.309418 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:08:44.309418 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:08:44.309418 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:08:44.309418 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:08:44.310659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:08:44.310659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:08:44.310659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:08:44.310659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:08:44.310659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:44.310659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:44.310659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:44.310659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:44.310659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:44.310659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:44.310659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:44.310659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:44.310659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:44.310659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:44.310659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:44.310659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:44.311604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:44.311604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:44.311604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:44.311604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:44.311604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:08:44.311604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:08:44.311604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:08:44.311604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:08:44.311604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:08:44.311604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:08:44.311604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:08:44.311604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:44.314639 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 269/549] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Jul 25 19:08:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:44.316450 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d1dd5-619b-48f1-b562-da1963a727c2 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:44.317100 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d1dd5-619b-48f1-b562-da1963a727c2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:44.317598 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d1dd5-619b-48f1-b562-da1963a727c2 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:44.323685 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 270/550] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:08:44.325869 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d1dd5-619b-48f1-b562-da1963a727c2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:44.362207 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:44.399547 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db0b1e-4abd-412d-aed4-65933bc8e2c8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:44.452632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d3e0eb-1fd0-4891-a8b0-30bf0461abd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:44.453742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3e0eb-1fd0-4891-a8b0-30bf0461abd0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:44.454558 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3e0eb-1fd0-4891-a8b0-30bf0461abd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:44.454852 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3e0eb-1fd0-4891-a8b0-30bf0461abd0 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:44.455943 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d3e0eb-1fd0-4891-a8b0-30bf0461abd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:44.456266 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d3e0eb-1fd0-4891-a8b0-30bf0461abd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:44.457752 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d3e0eb-1fd0-4891-a8b0-30bf0461abd0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:44.478351 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 281/551] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:08:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:44.480628 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3909eb-bd80-4f6a-8dc8-549b49580288 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:44.481667 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3909eb-bd80-4f6a-8dc8-549b49580288 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:44.482126 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3909eb-bd80-4f6a-8dc8-549b49580288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:44.506106 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6c3909eb-bd80-4f6a-8dc8-549b49580288 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:44.507408 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6c3909eb-bd80-4f6a-8dc8-549b49580288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:44.515291 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d1dd5-619b-48f1-b562-da1963a727c2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:44.572361 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4919a59-311a-45f5-94a0-92a63d409c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:44.573129 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4919a59-311a-45f5-94a0-92a63d409c53 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:44.573470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4919a59-311a-45f5-94a0-92a63d409c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:44.573755 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4919a59-311a-45f5-94a0-92a63d409c53 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:44.574644 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4919a59-311a-45f5-94a0-92a63d409c53 None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:44.575152 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4919a59-311a-45f5-94a0-92a63d409c53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:44.577067 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4919a59-311a-45f5-94a0-92a63d409c53 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:44.580125 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:44.596841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 271/552] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Jul 25 19:08:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:44.598523 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 282/553] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:08:44.598729 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4919a59-311a-45f5-94a0-92a63d409c53 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:44.600255 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3909eb-bd80-4f6a-8dc8-549b49580288 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:44.733193 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:44.755864 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3909eb-bd80-4f6a-8dc8-549b49580288 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:44.759929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4919a59-311a-45f5-94a0-92a63d409c53 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:44.801561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d2d93-b498-4237-ab36-ed844cda3d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:44.802771 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d2d93-b498-4237-ab36-ed844cda3d9a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:44.803259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d2d93-b498-4237-ab36-ed844cda3d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:44.803765 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d2d93-b498-4237-ab36-ed844cda3d9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:44.807415 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb4552-78be-4d1f-b540-753d414976a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:44.808133 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb4552-78be-4d1f-b540-753d414976a7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:44.808495 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb4552-78be-4d1f-b540-753d414976a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:44.808890 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bb4552-78be-4d1f-b540-753d414976a7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:44.818489 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14d2d93-b498-4237-ab36-ed844cda3d9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:44.819109 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bb4552-78be-4d1f-b540-753d414976a7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:44.820809 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bb4552-78be-4d1f-b540-753d414976a7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:44.821176 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14d2d93-b498-4237-ab36-ed844cda3d9a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:44.838659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-e9bb4552-78be-4d1f-b540-753d414976a7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 19:08:44.838659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:44.838659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:44.838659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:44.838659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:44.838659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:44.838659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:44.838659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:44.838659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:44.838659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:44.838659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:44.838659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:44.838659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:08:44.838659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:08:44.838659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:08:44.838659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:08:44.839760 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:44.839760 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:44.839760 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:08:44.839760 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:08:44.839760 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:08:44.839760 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:08:44.839760 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:44.839760 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:44.839760 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:44.839760 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:44.839760 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:44.839760 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:44.839760 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:44.839760 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:44.839760 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:44.839760 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:44.839760 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:44.839760 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:44.841641 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:44.841641 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:44.841641 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:08:44.841641 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:08:44.841641 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:08:44.841641 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:08:44.841641 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:08:44.841641 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:08:44.841641 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 19:08:44.841641 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:44.842734 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 272/554] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Jul 25 19:08:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:44.843716 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb4552-78be-4d1f-b540-753d414976a7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:44.848083 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-f14d2d93-b498-4237-ab36-ed844cda3d9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:44.848083 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:44.848083 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:44.848083 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:44.848083 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:44.848083 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:44.848083 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:44.848083 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:44.848083 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:44.848083 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:44.848083 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:44.848083 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:44.848083 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:08:44.848083 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:08:44.848083 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:08:44.849362 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:08:44.849362 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:44.849362 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:44.849362 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:44.849362 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:44.849362 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:44.849362 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:44.849362 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:44.849362 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:44.849362 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:44.849362 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:44.849362 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:44.849362 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:44.849362 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:44.849362 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:44.849362 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:44.849362 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:44.850458 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:08:44.850458 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:08:44.850458 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:44.850458 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:44.852857 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 283/555] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:08:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:44.854174 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d2d93-b498-4237-ab36-ed844cda3d9a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:44.924178 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:44.982357 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d2d93-b498-4237-ab36-ed844cda3d9a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:44.982927 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bb4552-78be-4d1f-b540-753d414976a7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:45.025541 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40b003-261a-451f-8815-facc7bb8d3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:45.026761 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40b003-261a-451f-8815-facc7bb8d3c0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:45.027281 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40b003-261a-451f-8815-facc7bb8d3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:45.028534 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40b003-261a-451f-8815-facc7bb8d3c0 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:45.029636 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa40b003-261a-451f-8815-facc7bb8d3c0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:45.030157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa40b003-261a-451f-8815-facc7bb8d3c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:45.032040 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa40b003-261a-451f-8815-facc7bb8d3c0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:45.032205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6cfb2a-3139-4047-bc53-b1fe5e29b87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:45.033114 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6cfb2a-3139-4047-bc53-b1fe5e29b87a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:45.033566 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6cfb2a-3139-4047-bc53-b1fe5e29b87a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:45.033969 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6cfb2a-3139-4047-bc53-b1fe5e29b87a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:45.048811 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6cfb2a-3139-4047-bc53-b1fe5e29b87a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:08:45.057838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-fb6cfb2a-3139-4047-bc53-b1fe5e29b87a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:08:45.057838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:45.057838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:45.057838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:45.057838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:45.057838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:45.057838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:45.057838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:45.057838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:45.057838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:45.057838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:45.057838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:45.057838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:08:45.057838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:08:45.057838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:08:45.057838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:08:45.058927 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:08:45.058927 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:08:45.058927 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:08:45.058927 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:08:45.058927 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:45.058927 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:45.058927 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:45.058927 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:45.058927 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:45.058927 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:45.058927 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:45.058927 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:45.058927 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:45.058927 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:45.058927 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:45.058927 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:45.060892 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:45.060892 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:45.060892 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:45.060892 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:45.060892 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:08:45.060892 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:08:45.060892 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:08:45.060892 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:08:45.060892 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:08:45.060892 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:08:45.060892 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:08:45.060892 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:45.062709 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 284/556] 10.222.0.18 () {56 vars in 1135 bytes} [Mon Jul 25 19:08:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:45.062709 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40b003-261a-451f-8815-facc7bb8d3c0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:45.064895 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 273/557] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Jul 25 19:08:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:45.066787 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6cfb2a-3139-4047-bc53-b1fe5e29b87a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:45.084151 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:45.263861 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40b003-261a-451f-8815-facc7bb8d3c0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:45.265727 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:45.292390 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6cfb2a-3139-4047-bc53-b1fe5e29b87a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:45.313788 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60986f78-0669-4a0d-a05e-f7e045dc40af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:45.314854 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60986f78-0669-4a0d-a05e-f7e045dc40af None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:45.315297 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60986f78-0669-4a0d-a05e-f7e045dc40af None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:45.315780 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60986f78-0669-4a0d-a05e-f7e045dc40af None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:45.316715 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60986f78-0669-4a0d-a05e-f7e045dc40af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:45.317188 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60986f78-0669-4a0d-a05e-f7e045dc40af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:45.319010 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60986f78-0669-4a0d-a05e-f7e045dc40af None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:45.343459 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 285/558] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:08:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:45.344775 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60986f78-0669-4a0d-a05e-f7e045dc40af None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:45.364462 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512bb5f-ecf2-4892-b0a0-99ba2bd27d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:45.365351 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512bb5f-ecf2-4892-b0a0-99ba2bd27d3e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:45.365850 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512bb5f-ecf2-4892-b0a0-99ba2bd27d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:45.366350 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512bb5f-ecf2-4892-b0a0-99ba2bd27d3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:45.490731 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60986f78-0669-4a0d-a05e-f7e045dc40af None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:45.508819 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512bb5f-ecf2-4892-b0a0-99ba2bd27d3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:45.512888 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512bb5f-ecf2-4892-b0a0-99ba2bd27d3e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:45.516908 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:45.545387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d7800-28e3-4dc2-8344-6c96532a7211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:45.546806 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d7800-28e3-4dc2-8344-6c96532a7211 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:45.547411 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d7800-28e3-4dc2-8344-6c96532a7211 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:45.547940 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d7800-28e3-4dc2-8344-6c96532a7211 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:45.549335 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d7800-28e3-4dc2-8344-6c96532a7211 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:45.549902 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d7800-28e3-4dc2-8344-6c96532a7211 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:45.551552 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d7800-28e3-4dc2-8344-6c96532a7211 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:45.564334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 286/559] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Jul 25 19:08:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:45.565555 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d7800-28e3-4dc2-8344-6c96532a7211 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:45.600319 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:45.704656 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d7800-28e3-4dc2-8344-6c96532a7211 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:45.743076 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 274/560] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:08:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 672 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 19:08:45.744908 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512bb5f-ecf2-4892-b0a0-99ba2bd27d3e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:45.755370 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12773997-c3d1-4068-be4d-3dfcd130eacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:45.756581 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12773997-c3d1-4068-be4d-3dfcd130eacc None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:45.759431 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12773997-c3d1-4068-be4d-3dfcd130eacc None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:45.759734 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12773997-c3d1-4068-be4d-3dfcd130eacc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:45.770171 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12773997-c3d1-4068-be4d-3dfcd130eacc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:45.771733 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12773997-c3d1-4068-be4d-3dfcd130eacc None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:45.788274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-12773997-c3d1-4068-be4d-3dfcd130eacc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 19:08:45.788274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:45.788274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:45.788274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:45.788274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:45.788274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:45.788274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:45.788274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:45.788274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:45.788274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:45.788274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:45.788274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:45.788274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:08:45.788274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:08:45.788274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:08:45.788274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:08:45.789962 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:45.789962 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:45.789962 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:08:45.789962 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:08:45.789962 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:08:45.789962 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:08:45.789962 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:45.789962 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:45.789962 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:45.789962 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:45.789962 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:45.789962 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:45.789962 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:45.789962 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:45.789962 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:45.789962 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:45.789962 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:45.789962 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:45.792761 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:45.792761 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:45.792761 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:08:45.792761 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:08:45.792761 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:08:45.792761 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:08:45.792761 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:08:45.792761 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:08:45.792761 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 19:08:45.792761 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:45.792761 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 287/561] 10.222.0.18 () {56 vars in 1093 bytes} [Mon Jul 25 19:08:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:45.792761 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12773997-c3d1-4068-be4d-3dfcd130eacc None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:45.906040 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:45.927176 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12773997-c3d1-4068-be4d-3dfcd130eacc None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:45.969711 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512bb5f-ecf2-4892-b0a0-99ba2bd27d3e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:45.972151 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dbac2d-da64-47ec-b3d2-a0163f06b35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:45.973082 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dbac2d-da64-47ec-b3d2-a0163f06b35d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:45.973444 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dbac2d-da64-47ec-b3d2-a0163f06b35d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:45.973799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dbac2d-da64-47ec-b3d2-a0163f06b35d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:45.975170 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:45.990292 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dbac2d-da64-47ec-b3d2-a0163f06b35d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:45.992235 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dbac2d-da64-47ec-b3d2-a0163f06b35d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:46.014259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-52dbac2d-da64-47ec-b3d2-a0163f06b35d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:46.014259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:46.014259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:46.014259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:46.014259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:46.014259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:46.014259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:46.014259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:46.014259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:46.014259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:46.014259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:46.014259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:46.014259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:08:46.014259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:08:46.014259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:08:46.016078 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:08:46.016078 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:46.016078 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:46.016078 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:46.016078 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:46.016078 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:46.016078 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:46.016078 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:46.016078 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:46.016078 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:46.016078 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:46.016078 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:46.016078 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:46.016078 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:46.016078 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:46.016078 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:46.016078 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:46.017877 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:08:46.017877 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:08:46.017877 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:46.017877 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:46.020782 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 288/562] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:08:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:46.021830 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dbac2d-da64-47ec-b3d2-a0163f06b35d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:46.053620 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175b7afc-08df-4c6b-892d-01d350c8b384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:46.055600 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b7afc-08df-4c6b-892d-01d350c8b384 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:46.056441 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b7afc-08df-4c6b-892d-01d350c8b384 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:46.057208 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b7afc-08df-4c6b-892d-01d350c8b384 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:46.079122 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175b7afc-08df-4c6b-892d-01d350c8b384 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:46.082340 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175b7afc-08df-4c6b-892d-01d350c8b384 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:46.119241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-175b7afc-08df-4c6b-892d-01d350c8b384 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:46.119241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:46.119241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:46.119241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:46.119241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:46.119241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:46.119241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:46.119241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:46.119241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:46.119241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:46.119241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:46.119241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:46.119241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:08:46.119241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:08:46.119241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:08:46.120998 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:08:46.120998 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:46.120998 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:46.120998 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:46.120998 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:46.120998 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:46.120998 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:46.120998 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:46.120998 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:46.120998 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:46.120998 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:46.120998 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:46.120998 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:46.120998 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:46.120998 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:46.120998 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:46.120998 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:46.122835 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:08:46.122835 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:08:46.122835 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:46.122835 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:46.126647 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 275/563] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:08:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:46.128711 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175b7afc-08df-4c6b-892d-01d350c8b384 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:46.152479 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dbac2d-da64-47ec-b3d2-a0163f06b35d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:46.198620 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002bd343-5483-4d8a-a648-93bd84441bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:46.199957 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002bd343-5483-4d8a-a648-93bd84441bfb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:46.200481 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002bd343-5483-4d8a-a648-93bd84441bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:46.200882 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002bd343-5483-4d8a-a648-93bd84441bfb None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:46.201985 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002bd343-5483-4d8a-a648-93bd84441bfb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:46.202532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002bd343-5483-4d8a-a648-93bd84441bfb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:46.203875 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002bd343-5483-4d8a-a648-93bd84441bfb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:46.208360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:46.229884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 289/564] 10.222.0.18 () {56 vars in 1133 bytes} [Mon Jul 25 19:08:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:46.231900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002bd343-5483-4d8a-a648-93bd84441bfb None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:46.335313 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175b7afc-08df-4c6b-892d-01d350c8b384 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:46.397764 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002bd343-5483-4d8a-a648-93bd84441bfb None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:46.404481 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6e311-075a-4ea3-9b7e-8a709a584efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:46.405222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6e311-075a-4ea3-9b7e-8a709a584efe None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:46.405507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6e311-075a-4ea3-9b7e-8a709a584efe None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:46.405779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6e311-075a-4ea3-9b7e-8a709a584efe None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:46.406533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c6e311-075a-4ea3-9b7e-8a709a584efe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:46.406846 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c6e311-075a-4ea3-9b7e-8a709a584efe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:46.408420 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c6e311-075a-4ea3-9b7e-8a709a584efe None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:46.432902 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 276/565] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:46.432902 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6e311-075a-4ea3-9b7e-8a709a584efe None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:46.457555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:46.482667 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd793568-b6cf-40f5-95de-56517ca12ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:46.484218 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd793568-b6cf-40f5-95de-56517ca12ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:46.484940 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd793568-b6cf-40f5-95de-56517ca12ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:46.485443 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd793568-b6cf-40f5-95de-56517ca12ca1 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:46.486938 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd793568-b6cf-40f5-95de-56517ca12ca1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:46.487478 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd793568-b6cf-40f5-95de-56517ca12ca1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:46.490138 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd793568-b6cf-40f5-95de-56517ca12ca1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:46.514169 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 290/566] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:46.523725 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f749eb9-7067-46ce-a4eb-cb6b8d0c35ff None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:46.524151 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f749eb9-7067-46ce-a4eb-cb6b8d0c35ff None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:46.524639 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f749eb9-7067-46ce-a4eb-cb6b8d0c35ff None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:46.527286 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 291/567] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:08:46.529606 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f749eb9-7067-46ce-a4eb-cb6b8d0c35ff None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:46.627902 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6e311-075a-4ea3-9b7e-8a709a584efe None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:46.644073 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:46.697256 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5436f0d6-3140-486d-9002-04867b858da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:46.698655 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436f0d6-3140-486d-9002-04867b858da0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:46.699153 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436f0d6-3140-486d-9002-04867b858da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:46.700917 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436f0d6-3140-486d-9002-04867b858da0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:46.714811 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f749eb9-7067-46ce-a4eb-cb6b8d0c35ff None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:46.721716 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5436f0d6-3140-486d-9002-04867b858da0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:46.724312 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5436f0d6-3140-486d-9002-04867b858da0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:46.745928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-5436f0d6-3140-486d-9002-04867b858da0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:46.745928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:46.745928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:46.745928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:46.745928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:46.745928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:46.745928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:46.745928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:46.745928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:46.745928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:46.745928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:46.745928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:46.745928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:08:46.745928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:08:46.745928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:08:46.748624 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:08:46.748624 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:46.748624 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:46.748624 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:46.748624 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:46.748624 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:46.748624 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:46.748624 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:46.748624 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:46.748624 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:46.748624 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:46.748624 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:46.748624 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:46.748624 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:46.748624 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:46.748624 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:46.748624 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:46.750381 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:08:46.750381 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:08:46.750381 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:46.750381 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:46.750381 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 277/568] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:08:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:46.750381 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5436f0d6-3140-486d-9002-04867b858da0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:46.772969 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df704a0-d804-4e2a-bd11-6ae16ae8e189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:46.773821 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df704a0-d804-4e2a-bd11-6ae16ae8e189 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:46.774115 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df704a0-d804-4e2a-bd11-6ae16ae8e189 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:46.774395 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df704a0-d804-4e2a-bd11-6ae16ae8e189 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:46.788692 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df704a0-d804-4e2a-bd11-6ae16ae8e189 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:46.791766 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df704a0-d804-4e2a-bd11-6ae16ae8e189 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:46.823222 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-0df704a0-d804-4e2a-bd11-6ae16ae8e189 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:08:46.823222 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:46.823222 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:46.823222 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:46.823222 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:46.823222 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:46.823222 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:46.823222 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:46.823222 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:46.823222 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:46.823222 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:46.823222 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:46.823222 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:08:46.823222 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:08:46.823222 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:08:46.825305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:08:46.825305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:46.825305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:46.825305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:46.825305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:46.825305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:46.825305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:46.825305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:46.825305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:46.825305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:46.825305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:46.825305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:46.825305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:46.825305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:46.825305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:46.825305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:46.825305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:46.831878 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:08:46.831878 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:08:46.831878 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:08:46.831878 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:08:46.831878 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:08:46.831878 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:08:46.831878 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:08:46.831878 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:46.831878 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 292/569] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:08:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:46.831878 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8049c52-6939-4c2d-bcda-ff22a2f2c9cb None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:46.831878 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8049c52-6939-4c2d-bcda-ff22a2f2c9cb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:46.832605 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8049c52-6939-4c2d-bcda-ff22a2f2c9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:46.849311 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:46.862200 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d8049c52-6939-4c2d-bcda-ff22a2f2c9cb None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:46.864715 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d8049c52-6939-4c2d-bcda-ff22a2f2c9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:46.898597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5436f0d6-3140-486d-9002-04867b858da0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:46.944362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d75920b-2312-4e72-9c21-a9ac309b0cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:46.945185 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75920b-2312-4e72-9c21-a9ac309b0cb8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:46.945469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75920b-2312-4e72-9c21-a9ac309b0cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:46.945821 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75920b-2312-4e72-9c21-a9ac309b0cb8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:46.954412 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 293/570] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:08:46.955116 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d75920b-2312-4e72-9c21-a9ac309b0cb8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:46.955350 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8049c52-6939-4c2d-bcda-ff22a2f2c9cb None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:46.956773 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d75920b-2312-4e72-9c21-a9ac309b0cb8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:46.981929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-8d75920b-2312-4e72-9c21-a9ac309b0cb8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 25 19:08:46.981929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:46.981929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:46.981929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:46.981929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:46.981929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:46.981929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:46.981929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:46.981929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:46.981929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:46.981929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:46.981929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:46.981929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:08:46.981929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:08:46.981929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:08:46.981929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:08:46.983158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:46.983158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:46.983158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:08:46.983158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:08:46.983158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:08:46.983158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:08:46.983158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:46.983158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:46.983158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:46.983158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:46.983158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:46.983158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:46.983158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:46.983158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:46.983158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:46.983158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:46.983158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:46.983158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:46.984571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:46.984571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:46.984571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:08:46.984571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:08:46.984571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:08:46.984571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:08:46.984571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:08:46.984571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:08:46.984571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 25 19:08:46.984571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:46.984571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 278/571] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:08:46 2022] GET /identity/v3/users/glance => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:46.984571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d75920b-2312-4e72-9c21-a9ac309b0cb8 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:47.014817 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:47.084082 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8049c52-6939-4c2d-bcda-ff22a2f2c9cb None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:47.114874 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d75920b-2312-4e72-9c21-a9ac309b0cb8 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:47.128141 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010a5d1-262d-4b69-aa05-7315c8a96c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:47.128841 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010a5d1-262d-4b69-aa05-7315c8a96c30 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:47.129147 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010a5d1-262d-4b69-aa05-7315c8a96c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:47.129424 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010a5d1-262d-4b69-aa05-7315c8a96c30 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:47.130190 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b010a5d1-262d-4b69-aa05-7315c8a96c30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:47.130567 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b010a5d1-262d-4b69-aa05-7315c8a96c30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:47.132018 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b010a5d1-262d-4b69-aa05-7315c8a96c30 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:47.146211 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 294/572] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:47.147729 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc8bae-43af-4735-9411-c23564ca58b5 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:47.148024 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc8bae-43af-4735-9411-c23564ca58b5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:47.148310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc8bae-43af-4735-9411-c23564ca58b5 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:47.149683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 295/573] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:08:47.150440 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc8bae-43af-4735-9411-c23564ca58b5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:47.158136 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:47.173027 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa604b51-b37b-4613-ae1a-62cb5c3c388c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:47.174170 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa604b51-b37b-4613-ae1a-62cb5c3c388c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:47.174666 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa604b51-b37b-4613-ae1a-62cb5c3c388c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:47.175223 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa604b51-b37b-4613-ae1a-62cb5c3c388c None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:47.176894 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa604b51-b37b-4613-ae1a-62cb5c3c388c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:47.177457 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa604b51-b37b-4613-ae1a-62cb5c3c388c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:47.179722 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa604b51-b37b-4613-ae1a-62cb5c3c388c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:47.194457 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 279/574] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Jul 25 19:08:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:47.194457 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa604b51-b37b-4613-ae1a-62cb5c3c388c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:47.285476 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc8bae-43af-4735-9411-c23564ca58b5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:47.289541 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:47.323823 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa604b51-b37b-4613-ae1a-62cb5c3c388c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:47.332240 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2057e350-40b4-4c30-a9c8-1bc830bec936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:47.333124 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057e350-40b4-4c30-a9c8-1bc830bec936 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:47.333576 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057e350-40b4-4c30-a9c8-1bc830bec936 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:47.333886 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057e350-40b4-4c30-a9c8-1bc830bec936 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:47.334748 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2057e350-40b4-4c30-a9c8-1bc830bec936 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:47.335059 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2057e350-40b4-4c30-a9c8-1bc830bec936 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:47.336833 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2057e350-40b4-4c30-a9c8-1bc830bec936 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:47.363194 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 296/575] 10.222.0.18 () {56 vars in 1137 bytes} [Mon Jul 25 19:08:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:47.365019 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2057e350-40b4-4c30-a9c8-1bc830bec936 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:47.377119 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a21233-b607-4890-b191-c7e33c1a1951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:47.378406 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a21233-b607-4890-b191-c7e33c1a1951 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:47.378855 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a21233-b607-4890-b191-c7e33c1a1951 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:47.379294 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a21233-b607-4890-b191-c7e33c1a1951 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:47.397830 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a21233-b607-4890-b191-c7e33c1a1951 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:08:47.411356 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-a9a21233-b607-4890-b191-c7e33c1a1951 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:08:47.411356 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:47.411356 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:47.411356 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:47.411356 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:47.411356 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:47.411356 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:47.411356 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:47.411356 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:47.411356 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:47.411356 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:47.411356 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:47.411356 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:08:47.411356 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:08:47.411356 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:08:47.411356 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:08:47.412459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:08:47.412459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:08:47.412459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:08:47.412459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:08:47.412459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:47.412459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:47.412459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:47.412459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:47.412459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:47.412459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:47.412459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:47.412459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:47.412459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:47.412459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:47.412459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:47.412459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:47.413346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:47.413346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:47.413346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:47.413346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:47.413346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:08:47.413346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:08:47.413346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:08:47.413346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:08:47.413346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:08:47.413346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:08:47.413346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:08:47.413346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:47.413997 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 280/576] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Jul 25 19:08:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:47.415013 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431a4a9-ca93-425d-baa8-f8fc3abd7c6a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:47.415370 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431a4a9-ca93-425d-baa8-f8fc3abd7c6a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:47.415819 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431a4a9-ca93-425d-baa8-f8fc3abd7c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:47.437671 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f431a4a9-ca93-425d-baa8-f8fc3abd7c6a None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:47.438939 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f431a4a9-ca93-425d-baa8-f8fc3abd7c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:47.492119 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:47.512871 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2057e350-40b4-4c30-a9c8-1bc830bec936 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:47.523428 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 281/577] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:08:47.524429 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f431a4a9-ca93-425d-baa8-f8fc3abd7c6a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:47.568496 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e0dcc-c4ad-452b-bfb6-ade73fe16428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:47.569417 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e0dcc-c4ad-452b-bfb6-ade73fe16428 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:47.569827 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e0dcc-c4ad-452b-bfb6-ade73fe16428 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:47.570118 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e0dcc-c4ad-452b-bfb6-ade73fe16428 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:47.579574 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e0dcc-c4ad-452b-bfb6-ade73fe16428 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:47.581124 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e0dcc-c4ad-452b-bfb6-ade73fe16428 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:47.596404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-918e0dcc-c4ad-452b-bfb6-ade73fe16428 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:08:47.596404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:47.596404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:47.596404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:47.596404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:47.596404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:47.596404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:47.596404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:47.596404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:47.596404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:47.596404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:47.596404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:47.596404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:08:47.596404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:08:47.596404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:08:47.598130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:08:47.598130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:47.598130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:47.598130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:47.598130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:47.598130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:47.598130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:47.598130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:47.598130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:47.598130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:47.598130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:47.598130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:47.598130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:47.598130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:47.598130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:47.598130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:47.598130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:47.606437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:08:47.606437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:08:47.606437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:08:47.606437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:08:47.606437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:08:47.606437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:08:47.606437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:08:47.606437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:47.606437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 297/578] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:08:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:47.606437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e0dcc-c4ad-452b-bfb6-ade73fe16428 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:47.638582 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:47.670876 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f431a4a9-ca93-425d-baa8-f8fc3abd7c6a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:47.731928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6a063-bba8-458b-9c0a-34f87ff29033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:47.733027 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6a063-bba8-458b-9c0a-34f87ff29033 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:47.733510 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6a063-bba8-458b-9c0a-34f87ff29033 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:47.733999 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6a063-bba8-458b-9c0a-34f87ff29033 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:47.735427 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae6a063-bba8-458b-9c0a-34f87ff29033 None admin] RBAC: Adding query filter params (role.id=23718d83b0864b16b0071e8ffe0b5861, scope.project.id=191069caccd04233b012585e29cfa18b, user.id=feafcf1b5a9844fe9738823fddd07e0e) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:47.736077 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae6a063-bba8-458b-9c0a-34f87ff29033 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:47.738557 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae6a063-bba8-458b-9c0a-34f87ff29033 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:47.745554 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e0dcc-c4ad-452b-bfb6-ade73fe16428 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:47.749223 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 282/579] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:08:47 2022] GET /identity/v3/role_assignments?user.id=feafcf1b5a9844fe9738823fddd07e0e&scope.project.id=191069caccd04233b012585e29cfa18b&role.id=23718d83b0864b16b0071e8ffe0b5861 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:47.750695 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6a063-bba8-458b-9c0a-34f87ff29033 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:47.791829 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba16cc-11a4-48c1-ad88-09adb13cb96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:47.793626 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba16cc-11a4-48c1-ad88-09adb13cb96e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:47.793992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba16cc-11a4-48c1-ad88-09adb13cb96e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:47.794290 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba16cc-11a4-48c1-ad88-09adb13cb96e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:47.813376 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddba16cc-11a4-48c1-ad88-09adb13cb96e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:47.816209 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddba16cc-11a4-48c1-ad88-09adb13cb96e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:47.821730 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:47.851844 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-ddba16cc-11a4-48c1-ad88-09adb13cb96e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:47.851844 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:47.851844 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:47.851844 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:47.851844 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:47.851844 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:47.851844 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:47.851844 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:47.851844 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:47.851844 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:47.851844 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:47.851844 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:47.851844 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:08:47.851844 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:08:47.851844 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:08:47.853725 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:08:47.853725 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:47.853725 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:47.853725 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:47.853725 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:47.853725 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:47.853725 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:47.853725 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:47.853725 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:47.853725 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:47.853725 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:47.853725 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:47.853725 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:47.853725 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:47.853725 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:47.853725 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:47.853725 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:47.856839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:08:47.856839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:08:47.856839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:47.856839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:47.862562 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 298/580] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:08:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:47.865079 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba16cc-11a4-48c1-ad88-09adb13cb96e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:47.908805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6a063-bba8-458b-9c0a-34f87ff29033 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:47.971156 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d985a96-979b-469a-8306-25b9243b2065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:47.972250 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d985a96-979b-469a-8306-25b9243b2065 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:47.972738 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d985a96-979b-469a-8306-25b9243b2065 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:47.973204 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d985a96-979b-469a-8306-25b9243b2065 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:47.974187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d985a96-979b-469a-8306-25b9243b2065 None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:47.974653 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d985a96-979b-469a-8306-25b9243b2065 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:47.976523 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d985a96-979b-469a-8306-25b9243b2065 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:47.988466 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 283/581] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Jul 25 19:08:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:47.989920 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d985a96-979b-469a-8306-25b9243b2065 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:48.009559 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:48.044065 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba16cc-11a4-48c1-ad88-09adb13cb96e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:48.092128 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5ed2fa-bdc3-46d6-af4e-122d7005cc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:48.093285 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5ed2fa-bdc3-46d6-af4e-122d7005cc6a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:48.093841 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5ed2fa-bdc3-46d6-af4e-122d7005cc6a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:48.094368 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5ed2fa-bdc3-46d6-af4e-122d7005cc6a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:48.119050 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5ed2fa-bdc3-46d6-af4e-122d7005cc6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:08:48.134661 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-2e5ed2fa-bdc3-46d6-af4e-122d7005cc6a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:08:48.134661 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:48.134661 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:48.134661 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:48.134661 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:48.134661 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:48.134661 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:48.134661 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:48.134661 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:48.134661 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:48.134661 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:48.134661 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:48.134661 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:08:48.134661 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:08:48.134661 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:08:48.134661 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:08:48.136157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:08:48.136157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:08:48.136157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:08:48.136157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:08:48.136157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:48.136157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:48.136157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:48.136157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:48.136157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:48.136157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:48.136157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:48.136157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:48.136157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:48.136157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:48.136157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:48.136157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:48.143116 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:48.143116 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:48.143116 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:48.143116 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:48.143116 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:08:48.143116 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:08:48.143116 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:08:48.143116 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:08:48.143116 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:08:48.143116 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:08:48.143116 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:08:48.143116 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:48.143116 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 299/582] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Jul 25 19:08:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:48.143116 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5ed2fa-bdc3-46d6-af4e-122d7005cc6a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:48.162192 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d985a96-979b-469a-8306-25b9243b2065 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:48.169735 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:48.227539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea84ea9-acae-4d65-ab29-96665bca6cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:48.228880 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea84ea9-acae-4d65-ab29-96665bca6cfa None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:48.228880 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea84ea9-acae-4d65-ab29-96665bca6cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:48.229658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea84ea9-acae-4d65-ab29-96665bca6cfa None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:48.230553 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea84ea9-acae-4d65-ab29-96665bca6cfa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:48.231067 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea84ea9-acae-4d65-ab29-96665bca6cfa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:48.232753 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea84ea9-acae-4d65-ab29-96665bca6cfa None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:48.259007 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 284/583] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Jul 25 19:08:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:48.259007 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea84ea9-acae-4d65-ab29-96665bca6cfa None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:48.306905 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5ed2fa-bdc3-46d6-af4e-122d7005cc6a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:48.380771 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b0ed9-b6f5-4f84-b93f-7c51ee810440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:48.381858 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b0ed9-b6f5-4f84-b93f-7c51ee810440 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:48.382421 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b0ed9-b6f5-4f84-b93f-7c51ee810440 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:48.382908 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b0ed9-b6f5-4f84-b93f-7c51ee810440 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:48.384341 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9b0ed9-b6f5-4f84-b93f-7c51ee810440 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:48.384878 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9b0ed9-b6f5-4f84-b93f-7c51ee810440 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:48.389006 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9b0ed9-b6f5-4f84-b93f-7c51ee810440 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:48.389919 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea84ea9-acae-4d65-ab29-96665bca6cfa None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:48.393003 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:48.414200 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 300/584] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:48.415750 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b0ed9-b6f5-4f84-b93f-7c51ee810440 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:48.460074 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847fb34-fe15-41be-8907-8c14d3e90652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:48.461433 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847fb34-fe15-41be-8907-8c14d3e90652 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:48.461913 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847fb34-fe15-41be-8907-8c14d3e90652 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:48.462209 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847fb34-fe15-41be-8907-8c14d3e90652 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:48.478583 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2847fb34-fe15-41be-8907-8c14d3e90652 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:48.480277 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2847fb34-fe15-41be-8907-8c14d3e90652 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:48.504981 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-2847fb34-fe15-41be-8907-8c14d3e90652 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:48.504981 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:48.504981 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:48.504981 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:48.504981 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:48.504981 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:48.504981 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:48.504981 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:48.504981 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:48.504981 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:48.504981 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:48.504981 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:48.504981 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:08:48.504981 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:08:48.504981 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:08:48.505966 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:08:48.505966 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:48.505966 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:48.505966 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:48.505966 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:48.505966 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:48.505966 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:48.505966 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:48.505966 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:48.505966 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:48.505966 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:48.505966 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:48.505966 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:48.505966 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:48.505966 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:48.505966 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:48.505966 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:48.507037 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:08:48.507037 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:08:48.507037 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:48.507037 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:48.507037 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 285/585] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:08:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:48.507397 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847fb34-fe15-41be-8907-8c14d3e90652 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:48.531052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:48.583140 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b0ed9-b6f5-4f84-b93f-7c51ee810440 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:48.632098 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747971c-f7db-415c-b90e-cf7964fc31cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:48.632704 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747971c-f7db-415c-b90e-cf7964fc31cc None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:48.633000 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747971c-f7db-415c-b90e-cf7964fc31cc None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:48.633294 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747971c-f7db-415c-b90e-cf7964fc31cc None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:48.634101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6747971c-f7db-415c-b90e-cf7964fc31cc None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:48.634411 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6747971c-f7db-415c-b90e-cf7964fc31cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:48.635925 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6747971c-f7db-415c-b90e-cf7964fc31cc None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:48.650914 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 301/586] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Jul 25 19:08:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:48.654053 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747971c-f7db-415c-b90e-cf7964fc31cc None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:48.669110 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847fb34-fe15-41be-8907-8c14d3e90652 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:48.723389 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758861d9-2dff-45da-98c3-d524f089b826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:48.724523 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758861d9-2dff-45da-98c3-d524f089b826 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:48.724913 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758861d9-2dff-45da-98c3-d524f089b826 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:48.725200 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758861d9-2dff-45da-98c3-d524f089b826 None admin] PATH_INFO: `/v3/projects/191069caccd04233b012585e29cfa18b/users/860e7d26ad364795bb25836aadd4ac7b/roles/23718d83b0864b16b0071e8ffe0b5861` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:48.741018 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:48.748488 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758861d9-2dff-45da-98c3-d524f089b826 None admin] RBAC: Authorizing `identity:create_grant(project_id=191069caccd04233b012585e29cfa18b, user_id=860e7d26ad364795bb25836aadd4ac7b, role_id=23718d83b0864b16b0071e8ffe0b5861)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:48.750159 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758861d9-2dff-45da-98c3-d524f089b826 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:48.778461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 286/587] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Jul 25 19:08:48 2022] PUT /identity/v3/projects/191069caccd04233b012585e29cfa18b/users/860e7d26ad364795bb25836aadd4ac7b/roles/23718d83b0864b16b0071e8ffe0b5861 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:08:48.779395 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758861d9-2dff-45da-98c3-d524f089b826 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:48.810286 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747971c-f7db-415c-b90e-cf7964fc31cc None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:48.858247 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c71147a-6c32-425c-80da-bcb3c80cf8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:48.859133 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71147a-6c32-425c-80da-bcb3c80cf8f7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:48.859582 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71147a-6c32-425c-80da-bcb3c80cf8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:48.859881 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71147a-6c32-425c-80da-bcb3c80cf8f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:48.869214 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c71147a-6c32-425c-80da-bcb3c80cf8f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:48.871030 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c71147a-6c32-425c-80da-bcb3c80cf8f7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:48.890760 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-5c71147a-6c32-425c-80da-bcb3c80cf8f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:08:48.890760 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:48.890760 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:48.890760 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:48.890760 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:48.890760 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:48.890760 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:48.890760 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:48.890760 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:48.890760 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:48.890760 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:48.890760 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:48.890760 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:08:48.890760 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:08:48.890760 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:08:48.892428 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:08:48.892428 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:48.892428 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:48.892428 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:48.892428 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:48.892428 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:48.892428 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:48.892428 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:48.892428 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:48.892428 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:48.892428 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:48.892428 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:48.892428 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:48.892428 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:48.892428 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:48.892428 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:48.892428 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:48.894521 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:08:48.894521 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:08:48.894521 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:08:48.894521 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:08:48.894521 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:08:48.894521 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:08:48.894521 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:08:48.894521 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:48.897662 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 302/588] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:08:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:48.898188 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c71147a-6c32-425c-80da-bcb3c80cf8f7 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:48.933268 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:49.015827 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758861d9-2dff-45da-98c3-d524f089b826 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:49.082898 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c71147a-6c32-425c-80da-bcb3c80cf8f7 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:49.101446 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac1ef7f-c93f-466e-b7ae-7042c75cda7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:49.102552 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1ef7f-c93f-466e-b7ae-7042c75cda7b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:49.103056 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1ef7f-c93f-466e-b7ae-7042c75cda7b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:49.103613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1ef7f-c93f-466e-b7ae-7042c75cda7b None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:49.104704 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac1ef7f-c93f-466e-b7ae-7042c75cda7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:49.105321 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac1ef7f-c93f-466e-b7ae-7042c75cda7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:49.107824 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac1ef7f-c93f-466e-b7ae-7042c75cda7b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:49.116447 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:49.146132 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 287/589] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:49.146132 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac1ef7f-c93f-466e-b7ae-7042c75cda7b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:49.147489 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d457f-b646-4721-b8af-c4d61a9bb4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:49.148755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d457f-b646-4721-b8af-c4d61a9bb4f6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:49.149266 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d457f-b646-4721-b8af-c4d61a9bb4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:49.149827 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d457f-b646-4721-b8af-c4d61a9bb4f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:49.178852 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113d457f-b646-4721-b8af-c4d61a9bb4f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:49.183136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113d457f-b646-4721-b8af-c4d61a9bb4f6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:49.214380 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-113d457f-b646-4721-b8af-c4d61a9bb4f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:49.214380 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:49.214380 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:49.214380 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:49.214380 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:49.214380 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:49.214380 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:49.214380 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:49.214380 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:49.214380 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:49.214380 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:49.214380 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:49.214380 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:08:49.214380 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:08:49.214380 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:08:49.215941 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:08:49.215941 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:49.215941 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:49.215941 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:49.215941 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:49.215941 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:49.215941 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:49.215941 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:49.215941 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:49.215941 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:49.215941 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:49.215941 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:49.215941 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:49.215941 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:49.215941 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:49.215941 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:49.215941 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:49.217375 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:08:49.217375 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:08:49.217375 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:49.217375 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:49.224611 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 303/590] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:08:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:49.226374 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d457f-b646-4721-b8af-c4d61a9bb4f6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:49.395730 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac1ef7f-c93f-466e-b7ae-7042c75cda7b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:49.417270 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d457f-b646-4721-b8af-c4d61a9bb4f6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:49.447536 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:49.464717 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced77f7-3881-4968-8bc3-633f0eefb320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:49.465617 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced77f7-3881-4968-8bc3-633f0eefb320 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:49.466009 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced77f7-3881-4968-8bc3-633f0eefb320 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:49.466657 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced77f7-3881-4968-8bc3-633f0eefb320 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:49.467743 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ced77f7-3881-4968-8bc3-633f0eefb320 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:49.468193 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ced77f7-3881-4968-8bc3-633f0eefb320 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:49.470273 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ced77f7-3881-4968-8bc3-633f0eefb320 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:49.491945 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 288/591] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Jul 25 19:08:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:49.493384 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced77f7-3881-4968-8bc3-633f0eefb320 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:49.502153 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac75605-0484-48e0-800e-89dc252ae831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:49.503155 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac75605-0484-48e0-800e-89dc252ae831 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:49.503719 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac75605-0484-48e0-800e-89dc252ae831 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:49.504133 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac75605-0484-48e0-800e-89dc252ae831 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:49.515431 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac75605-0484-48e0-800e-89dc252ae831 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:49.517138 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac75605-0484-48e0-800e-89dc252ae831 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:49.538685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-8ac75605-0484-48e0-800e-89dc252ae831 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 19:08:49.538685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:49.538685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:49.538685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:49.538685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:49.538685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:49.538685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:49.538685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:49.538685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:49.538685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:49.538685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:49.538685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:49.538685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:08:49.538685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:08:49.538685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:08:49.538685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:08:49.540741 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:49.540741 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:49.540741 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:08:49.540741 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:08:49.540741 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:08:49.540741 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:08:49.540741 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:49.540741 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:49.540741 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:49.540741 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:49.540741 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:49.540741 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:49.540741 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:49.540741 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:49.540741 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:49.540741 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:49.540741 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:49.540741 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:49.542826 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:49.542826 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:49.542826 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:08:49.542826 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:08:49.542826 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:08:49.542826 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:08:49.542826 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:08:49.542826 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:08:49.542826 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 19:08:49.542826 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:49.548333 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 304/592] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:08:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:49.549722 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac75605-0484-48e0-800e-89dc252ae831 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:49.602443 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:49.704575 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced77f7-3881-4968-8bc3-633f0eefb320 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:49.735290 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac75605-0484-48e0-800e-89dc252ae831 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:49.778034 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041156f1-e3ab-4e8f-bae8-82ec4c9a8ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:49.779762 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041156f1-e3ab-4e8f-bae8-82ec4c9a8ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:49.780850 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041156f1-e3ab-4e8f-bae8-82ec4c9a8ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:49.781663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041156f1-e3ab-4e8f-bae8-82ec4c9a8ad0 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:49.784116 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041156f1-e3ab-4e8f-bae8-82ec4c9a8ad0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:49.785292 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041156f1-e3ab-4e8f-bae8-82ec4c9a8ad0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:49.785824 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e501bc4-b4a4-4217-b6b7-df6e470d2036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:49.786907 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e501bc4-b4a4-4217-b6b7-df6e470d2036 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:49.787258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e501bc4-b4a4-4217-b6b7-df6e470d2036 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:49.787668 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e501bc4-b4a4-4217-b6b7-df6e470d2036 None admin] PATH_INFO: `/v3/projects/191069caccd04233b012585e29cfa18b/users/98560723b49940d38a6a5e9cf3f5d3b4/roles/23718d83b0864b16b0071e8ffe0b5861` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:49.789748 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041156f1-e3ab-4e8f-bae8-82ec4c9a8ad0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:49.806578 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e501bc4-b4a4-4217-b6b7-df6e470d2036 None admin] RBAC: Authorizing `identity:create_grant(project_id=191069caccd04233b012585e29cfa18b, user_id=98560723b49940d38a6a5e9cf3f5d3b4, role_id=23718d83b0864b16b0071e8ffe0b5861)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:49.809634 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e501bc4-b4a4-4217-b6b7-df6e470d2036 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:49.816713 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 289/593] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:08:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:49.818836 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041156f1-e3ab-4e8f-bae8-82ec4c9a8ad0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:49.839830 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 305/594] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Jul 25 19:08:49 2022] PUT /identity/v3/projects/191069caccd04233b012585e29cfa18b/users/98560723b49940d38a6a5e9cf3f5d3b4/roles/23718d83b0864b16b0071e8ffe0b5861 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:08:49.841537 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e501bc4-b4a4-4217-b6b7-df6e470d2036 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:49.948667 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:50.073245 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:50.077585 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041156f1-e3ab-4e8f-bae8-82ec4c9a8ad0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:50.132560 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd05540c-efda-4d59-93cc-e3fe651590b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:50.133371 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05540c-efda-4d59-93cc-e3fe651590b1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:50.133660 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05540c-efda-4d59-93cc-e3fe651590b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:50.133939 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05540c-efda-4d59-93cc-e3fe651590b1 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:50.134824 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd05540c-efda-4d59-93cc-e3fe651590b1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:50.135225 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd05540c-efda-4d59-93cc-e3fe651590b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:50.137205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd05540c-efda-4d59-93cc-e3fe651590b1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:50.161412 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e501bc4-b4a4-4217-b6b7-df6e470d2036 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:50.171113 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 290/595] 10.222.0.18 () {56 vars in 1137 bytes} [Mon Jul 25 19:08:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:50.172166 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd05540c-efda-4d59-93cc-e3fe651590b1 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:50.258619 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5ba8f-d872-4c60-946d-cd000fe9e3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:50.260899 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5ba8f-d872-4c60-946d-cd000fe9e3b5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:50.262709 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5ba8f-d872-4c60-946d-cd000fe9e3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:50.263629 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5ba8f-d872-4c60-946d-cd000fe9e3b5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:50.281443 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb5ba8f-d872-4c60-946d-cd000fe9e3b5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:50.285166 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb5ba8f-d872-4c60-946d-cd000fe9e3b5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:50.322637 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-dbb5ba8f-d872-4c60-946d-cd000fe9e3b5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 19:08:50.322637 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:50.322637 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:50.322637 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:50.322637 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:50.322637 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:50.322637 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:50.322637 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:50.322637 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:50.322637 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:50.322637 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:50.322637 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:50.322637 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:08:50.322637 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:08:50.322637 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:08:50.322637 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:08:50.325755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:50.325755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:50.325755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:08:50.325755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:08:50.325755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:08:50.325755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:08:50.325755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:50.325755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:50.325755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:50.325755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:50.325755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:50.325755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:50.325755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:50.325755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:50.325755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:50.325755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:50.325755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:50.325755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:50.327759 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:50.327759 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:50.327759 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:08:50.327759 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:08:50.327759 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:08:50.327759 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:08:50.327759 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:08:50.327759 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:08:50.327759 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 19:08:50.327759 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:50.333035 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 306/596] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Jul 25 19:08:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 492 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:50.343641 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5ba8f-d872-4c60-946d-cd000fe9e3b5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:50.515912 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd05540c-efda-4d59-93cc-e3fe651590b1 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:50.519781 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:50.611905 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5ba8f-d872-4c60-946d-cd000fe9e3b5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:50.690876 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9c769-ee85-4b96-8555-1ca68c6800bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:50.691712 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdce34b-fa27-48d5-a2aa-d703fe4f4efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:50.692334 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9c769-ee85-4b96-8555-1ca68c6800bc None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:50.692850 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9c769-ee85-4b96-8555-1ca68c6800bc None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:50.693352 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9c769-ee85-4b96-8555-1ca68c6800bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:50.694148 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdce34b-fa27-48d5-a2aa-d703fe4f4efd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:50.694982 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdce34b-fa27-48d5-a2aa-d703fe4f4efd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:50.695849 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdce34b-fa27-48d5-a2aa-d703fe4f4efd None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:50.698110 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdce34b-fa27-48d5-a2aa-d703fe4f4efd None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:50.698981 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdce34b-fa27-48d5-a2aa-d703fe4f4efd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:50.703740 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdce34b-fa27-48d5-a2aa-d703fe4f4efd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:50.723044 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:50.747237 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 307/597] 10.222.0.18 () {56 vars in 1139 bytes} [Mon Jul 25 19:08:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:50.748685 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac9c769-ee85-4b96-8555-1ca68c6800bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:50.751538 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac9c769-ee85-4b96-8555-1ca68c6800bc None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:50.753283 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdce34b-fa27-48d5-a2aa-d703fe4f4efd None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:50.820602 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-9ac9c769-ee85-4b96-8555-1ca68c6800bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:50.820602 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:50.820602 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:50.820602 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:50.820602 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:50.820602 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:50.820602 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:50.820602 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:50.820602 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:50.820602 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:50.820602 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:50.820602 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:50.820602 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:08:50.820602 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:08:50.820602 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:08:50.822324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:08:50.822324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:50.822324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:50.822324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:50.822324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:50.822324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:50.822324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:50.822324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:50.822324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:50.822324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:50.822324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:50.822324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:50.822324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:50.822324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:50.822324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:50.822324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:50.822324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:50.825430 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:08:50.825430 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:08:50.825430 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:50.825430 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:50.831465 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 291/598] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:08:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 660 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:50.838367 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9c769-ee85-4b96-8555-1ca68c6800bc None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:51.042126 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdce34b-fa27-48d5-a2aa-d703fe4f4efd None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:51.066313 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9c769-ee85-4b96-8555-1ca68c6800bc None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:51.112521 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4cb6d4-629e-479e-a6b3-e6a7bf6b92cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:51.114035 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4cb6d4-629e-479e-a6b3-e6a7bf6b92cb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:51.114400 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4cb6d4-629e-479e-a6b3-e6a7bf6b92cb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:51.114908 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4cb6d4-629e-479e-a6b3-e6a7bf6b92cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:51.136220 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4cb6d4-629e-479e-a6b3-e6a7bf6b92cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:51.139676 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4cb6d4-629e-479e-a6b3-e6a7bf6b92cb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:51.141700 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a440b-9fe4-4b5a-a57d-f4b795a17789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:51.143039 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a440b-9fe4-4b5a-a57d-f4b795a17789 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:51.143803 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a440b-9fe4-4b5a-a57d-f4b795a17789 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:51.144515 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a440b-9fe4-4b5a-a57d-f4b795a17789 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:51.145991 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712a440b-9fe4-4b5a-a57d-f4b795a17789 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:51.146760 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712a440b-9fe4-4b5a-a57d-f4b795a17789 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:51.149675 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712a440b-9fe4-4b5a-a57d-f4b795a17789 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:51.174916 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 292/599] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:08:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:51.177759 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0722ce-64a3-472e-b370-cb826bad3437 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:51.178620 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0722ce-64a3-472e-b370-cb826bad3437 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:51.179380 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0722ce-64a3-472e-b370-cb826bad3437 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:51.181698 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-7d4cb6d4-629e-479e-a6b3-e6a7bf6b92cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:51.181698 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:51.181698 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:51.181698 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:51.181698 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:51.181698 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:51.181698 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:51.181698 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:51.181698 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:51.181698 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:51.181698 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:51.181698 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:51.181698 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:08:51.181698 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:08:51.181698 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:08:51.183315 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:08:51.183315 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:51.183315 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:51.183315 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:51.183315 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:51.183315 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:51.183315 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:51.183315 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:51.183315 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:51.183315 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:51.183315 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:51.183315 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:51.183315 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:51.183315 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:51.183315 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:51.183315 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:51.183315 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:51.186507 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:08:51.186507 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:08:51.186507 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:51.186507 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:51.186507 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 308/600] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:08:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 433 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:08:51.186999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4cb6d4-629e-479e-a6b3-e6a7bf6b92cb None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:51.188805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 293/601] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:51 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:08:51.195375 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0722ce-64a3-472e-b370-cb826bad3437 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:51.239984 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:51.297043 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:51.403933 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4cb6d4-629e-479e-a6b3-e6a7bf6b92cb None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:51.425150 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0722ce-64a3-472e-b370-cb826bad3437 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:51.474641 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbed127e-ab96-4c8e-88b1-483952741878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:51.476114 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed127e-ab96-4c8e-88b1-483952741878 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:51.476611 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed127e-ab96-4c8e-88b1-483952741878 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:51.477096 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed127e-ab96-4c8e-88b1-483952741878 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:51.489038 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551489e-66d5-4ac5-81ae-8dafc7666c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:51.490101 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551489e-66d5-4ac5-81ae-8dafc7666c38 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:51.490624 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551489e-66d5-4ac5-81ae-8dafc7666c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:51.490732 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbed127e-ab96-4c8e-88b1-483952741878 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:51.491419 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551489e-66d5-4ac5-81ae-8dafc7666c38 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:51.492636 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0551489e-66d5-4ac5-81ae-8dafc7666c38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:51.493193 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0551489e-66d5-4ac5-81ae-8dafc7666c38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:51.493752 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbed127e-ab96-4c8e-88b1-483952741878 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:51.495266 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0551489e-66d5-4ac5-81ae-8dafc7666c38 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:51.516593 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 294/602] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:08:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:51.518811 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41299f98-1edb-42db-a84c-da274d3ec684 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:51.519420 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41299f98-1edb-42db-a84c-da274d3ec684 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:51.519899 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41299f98-1edb-42db-a84c-da274d3ec684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:51.521594 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-dbed127e-ab96-4c8e-88b1-483952741878 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:08:51.521594 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:51.521594 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:51.521594 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:51.521594 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:51.521594 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:51.521594 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:51.521594 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:51.521594 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:51.521594 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:51.521594 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:51.521594 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:51.521594 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:08:51.521594 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:08:51.521594 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:08:51.522519 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:08:51.522519 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:51.522519 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:51.522519 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:51.522519 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:51.522519 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:51.522519 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:51.522519 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:51.522519 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:51.522519 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:51.522519 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:51.522519 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:51.522519 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:51.522519 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:51.522519 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:51.522519 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:51.522519 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:51.523481 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:08:51.523481 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:08:51.523481 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:08:51.523481 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:08:51.523481 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:08:51.523481 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:08:51.523481 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:08:51.523481 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:51.525088 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 309/603] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:08:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:51.526895 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbed127e-ab96-4c8e-88b1-483952741878 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:51.548237 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-41299f98-1edb-42db-a84c-da274d3ec684 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:51.550104 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-41299f98-1edb-42db-a84c-da274d3ec684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:51.644613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 295/604] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:08:51.645800 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41299f98-1edb-42db-a84c-da274d3ec684 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:51.650032 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:51.716620 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbed127e-ab96-4c8e-88b1-483952741878 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:51.732822 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:51.765837 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51efa54b-5f9f-4cf1-8295-b18f436899e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:51.766858 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51efa54b-5f9f-4cf1-8295-b18f436899e0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:51.767239 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51efa54b-5f9f-4cf1-8295-b18f436899e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:51.767621 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51efa54b-5f9f-4cf1-8295-b18f436899e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:51.776439 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51efa54b-5f9f-4cf1-8295-b18f436899e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:51.778494 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51efa54b-5f9f-4cf1-8295-b18f436899e0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:51.791019 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41299f98-1edb-42db-a84c-da274d3ec684 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:51.804884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-51efa54b-5f9f-4cf1-8295-b18f436899e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:08:51.804884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:51.804884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:51.804884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:51.804884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:51.804884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:51.804884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:51.804884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:51.804884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:51.804884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:51.804884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:51.804884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:51.804884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:08:51.804884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:08:51.804884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:08:51.806001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:08:51.806001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:51.806001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:51.806001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:51.806001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:51.806001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:51.806001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:51.806001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:51.806001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:51.806001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:51.806001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:51.806001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:51.806001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:51.806001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:51.806001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:51.806001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:51.806001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:51.806896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:08:51.806896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:08:51.806896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:08:51.806896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:08:51.806896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:08:51.806896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:08:51.806896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:08:51.806896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:51.808085 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 310/605] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:08:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:51.811762 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690ad09-32b7-42cb-bcb1-3ccbcdd1711a None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:51.812280 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690ad09-32b7-42cb-bcb1-3ccbcdd1711a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:51.812790 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690ad09-32b7-42cb-bcb1-3ccbcdd1711a None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:51.815395 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 311/606] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:08:51.816861 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5690ad09-32b7-42cb-bcb1-3ccbcdd1711a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:51.844533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44720450-c7f1-4a7b-ade3-77f03003f1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:51.845387 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44720450-c7f1-4a7b-ade3-77f03003f1f9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:51.845697 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44720450-c7f1-4a7b-ade3-77f03003f1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:51.846002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44720450-c7f1-4a7b-ade3-77f03003f1f9 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:51.846760 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44720450-c7f1-4a7b-ade3-77f03003f1f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:51.847070 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44720450-c7f1-4a7b-ade3-77f03003f1f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:51.848585 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44720450-c7f1-4a7b-ade3-77f03003f1f9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:51.862658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 296/607] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Jul 25 19:08:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:51.863571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44720450-c7f1-4a7b-ade3-77f03003f1f9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:51.965473 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:52.031846 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5690ad09-32b7-42cb-bcb1-3ccbcdd1711a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:52.035618 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44720450-c7f1-4a7b-ade3-77f03003f1f9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:52.095220 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92360be-61a4-4032-9233-b85b3b7e2c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:52.096090 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6023c9-9c21-43c7-abad-7e0c1fb41d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:52.096877 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92360be-61a4-4032-9233-b85b3b7e2c4e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:52.096877 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92360be-61a4-4032-9233-b85b3b7e2c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:52.097147 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92360be-61a4-4032-9233-b85b3b7e2c4e None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:52.097975 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6023c9-9c21-43c7-abad-7e0c1fb41d6b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:52.098256 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92360be-61a4-4032-9233-b85b3b7e2c4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:52.098411 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6023c9-9c21-43c7-abad-7e0c1fb41d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:52.098545 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6023c9-9c21-43c7-abad-7e0c1fb41d6b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:52.098829 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92360be-61a4-4032-9233-b85b3b7e2c4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:52.100668 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92360be-61a4-4032-9233-b85b3b7e2c4e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:52.116798 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6023c9-9c21-43c7-abad-7e0c1fb41d6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:08:52.123646 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 297/608] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Jul 25 19:08:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:52.125804 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17fe9f5-c566-44ea-9799-b10033758e6a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:52.126272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17fe9f5-c566-44ea-9799-b10033758e6a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:52.126738 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17fe9f5-c566-44ea-9799-b10033758e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:52.132884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-7f6023c9-9c21-43c7-abad-7e0c1fb41d6b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:08:52.132884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:52.132884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:52.132884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:52.132884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:52.132884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:52.132884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:52.132884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:52.132884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:52.132884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:52.132884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:52.132884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:52.132884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:08:52.132884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:08:52.132884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:08:52.132884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:08:52.134870 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:08:52.134870 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:08:52.134870 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:08:52.134870 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:08:52.134870 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:52.134870 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:52.134870 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:52.134870 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:52.134870 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:52.134870 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:52.134870 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:52.134870 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:52.134870 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:52.134870 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:52.134870 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:52.134870 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:52.143961 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:52.143961 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:52.143961 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:52.143961 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:52.143961 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:08:52.143961 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:08:52.143961 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:08:52.143961 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:08:52.143961 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:08:52.143961 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:08:52.143961 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:08:52.143961 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:52.143961 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 312/609] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Jul 25 19:08:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:52.143961 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6023c9-9c21-43c7-abad-7e0c1fb41d6b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:52.155571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f17fe9f5-c566-44ea-9799-b10033758e6a None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:52.157455 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f17fe9f5-c566-44ea-9799-b10033758e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:52.184179 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:52.259148 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 298/610] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:08:52.261080 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17fe9f5-c566-44ea-9799-b10033758e6a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:52.304967 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6023c9-9c21-43c7-abad-7e0c1fb41d6b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:52.365337 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3eb34-426a-41a2-a296-e96b5ad1fb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:52.366698 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3eb34-426a-41a2-a296-e96b5ad1fb83 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:52.367182 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3eb34-426a-41a2-a296-e96b5ad1fb83 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:52.367746 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3eb34-426a-41a2-a296-e96b5ad1fb83 None admin] PATH_INFO: `/v3/projects/191069caccd04233b012585e29cfa18b/users/2aa720a35b9844848abcc42a4c4d6a2d/roles/23718d83b0864b16b0071e8ffe0b5861` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:52.370597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:52.392576 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d3eb34-426a-41a2-a296-e96b5ad1fb83 None admin] RBAC: Authorizing `identity:create_grant(project_id=191069caccd04233b012585e29cfa18b, user_id=2aa720a35b9844848abcc42a4c4d6a2d, role_id=23718d83b0864b16b0071e8ffe0b5861)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:52.394291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d3eb34-426a-41a2-a296-e96b5ad1fb83 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:52.420130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 313/611] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Jul 25 19:08:52 2022] PUT /identity/v3/projects/191069caccd04233b012585e29cfa18b/users/2aa720a35b9844848abcc42a4c4d6a2d/roles/23718d83b0864b16b0071e8ffe0b5861 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:08:52.421595 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3eb34-426a-41a2-a296-e96b5ad1fb83 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:52.455924 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17fe9f5-c566-44ea-9799-b10033758e6a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:52.511076 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c76cf-e8cd-4c2a-ab96-a957b254cc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:52.512592 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c76cf-e8cd-4c2a-ab96-a957b254cc7a None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:52.513209 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c76cf-e8cd-4c2a-ab96-a957b254cc7a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:52.513702 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c76cf-e8cd-4c2a-ab96-a957b254cc7a None admin] PATH_INFO: `/v3/projects/191069caccd04233b012585e29cfa18b/users/181c84715c094100b414cea3c4919eb7/roles/23718d83b0864b16b0071e8ffe0b5861` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:52.549964 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c76cf-e8cd-4c2a-ab96-a957b254cc7a None admin] RBAC: Authorizing `identity:create_grant(project_id=191069caccd04233b012585e29cfa18b, user_id=181c84715c094100b414cea3c4919eb7, role_id=23718d83b0864b16b0071e8ffe0b5861)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:52.551699 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c76cf-e8cd-4c2a-ab96-a957b254cc7a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:52.575388 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 299/612] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Jul 25 19:08:52 2022] PUT /identity/v3/projects/191069caccd04233b012585e29cfa18b/users/181c84715c094100b414cea3c4919eb7/roles/23718d83b0864b16b0071e8ffe0b5861 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:08:52.576746 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c76cf-e8cd-4c2a-ab96-a957b254cc7a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:52.604499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:52.649059 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3eb34-426a-41a2-a296-e96b5ad1fb83 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:52.744609 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce9daa-7201-448e-9361-0096bca32233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:52.746046 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce9daa-7201-448e-9361-0096bca32233 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:52.746558 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce9daa-7201-448e-9361-0096bca32233 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:52.747171 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce9daa-7201-448e-9361-0096bca32233 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:52.747973 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c76cf-e8cd-4c2a-ab96-a957b254cc7a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:52.750054 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ce9daa-7201-448e-9361-0096bca32233 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:52.750714 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ce9daa-7201-448e-9361-0096bca32233 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:52.752280 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:52.753362 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ce9daa-7201-448e-9361-0096bca32233 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:52.779601 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 314/613] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Jul 25 19:08:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:52.779601 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d8f0f-b73b-4d90-bd80-aa505f23b1ff None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:52.779601 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d8f0f-b73b-4d90-bd80-aa505f23b1ff None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:52.779601 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d8f0f-b73b-4d90-bd80-aa505f23b1ff None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:52.782458 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 315/614] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:08:52.784136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216d8f0f-b73b-4d90-bd80-aa505f23b1ff None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:52.825604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef85fe4-4d63-41b1-8cc1-ca53238d83eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:52.826547 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef85fe4-4d63-41b1-8cc1-ca53238d83eb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:52.826963 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef85fe4-4d63-41b1-8cc1-ca53238d83eb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:52.827278 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef85fe4-4d63-41b1-8cc1-ca53238d83eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:52.848130 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef85fe4-4d63-41b1-8cc1-ca53238d83eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:08:52.859646 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-4ef85fe4-4d63-41b1-8cc1-ca53238d83eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:08:52.859646 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:52.859646 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:52.859646 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:52.859646 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:52.859646 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:52.859646 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:52.859646 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:52.859646 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:52.859646 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:52.859646 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:52.859646 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:52.859646 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:08:52.859646 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:08:52.859646 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:08:52.859646 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:08:52.860729 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:08:52.860729 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:08:52.860729 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:08:52.860729 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:08:52.860729 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:52.860729 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:52.860729 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:52.860729 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:52.860729 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:52.860729 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:52.860729 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:52.860729 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:52.860729 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:52.860729 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:52.860729 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:52.860729 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:52.861743 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:52.861743 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:52.861743 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:52.861743 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:52.861743 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:08:52.861743 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:08:52.861743 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:08:52.861743 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:08:52.861743 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:08:52.861743 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:08:52.861743 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:08:52.861743 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:52.867035 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 300/615] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Jul 25 19:08:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:52.869418 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32556784-cca0-411a-ba04-b22dd4f54f2c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:52.869725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32556784-cca0-411a-ba04-b22dd4f54f2c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:52.870005 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32556784-cca0-411a-ba04-b22dd4f54f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:52.895798 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-32556784-cca0-411a-ba04-b22dd4f54f2c None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:52.897076 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-32556784-cca0-411a-ba04-b22dd4f54f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:53.015748 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 301/616] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:08:53.017836 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32556784-cca0-411a-ba04-b22dd4f54f2c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:53.050364 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216d8f0f-b73b-4d90-bd80-aa505f23b1ff None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:53.128809 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44466787-e654-4b86-a2fd-b521eaf34307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:53.130147 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44466787-e654-4b86-a2fd-b521eaf34307 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:53.130640 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44466787-e654-4b86-a2fd-b521eaf34307 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:53.131077 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44466787-e654-4b86-a2fd-b521eaf34307 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:53.147886 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:53.154425 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44466787-e654-4b86-a2fd-b521eaf34307 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:53.157606 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44466787-e654-4b86-a2fd-b521eaf34307 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:53.175020 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:53.185531 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32556784-cca0-411a-ba04-b22dd4f54f2c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:53.193379 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-44466787-e654-4b86-a2fd-b521eaf34307 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:53.193379 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:53.193379 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:53.193379 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:53.193379 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:53.193379 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:53.193379 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:53.193379 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:53.193379 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:53.193379 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:53.193379 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:53.193379 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:53.193379 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:08:53.193379 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:08:53.193379 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:08:53.194526 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:08:53.194526 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:53.194526 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:53.194526 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:53.194526 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:53.194526 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:53.194526 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:53.194526 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:53.194526 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:53.194526 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:53.194526 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:53.194526 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:53.194526 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:53.194526 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:53.194526 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:53.194526 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:53.194526 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:53.195544 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:08:53.195544 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:08:53.195544 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:53.195544 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:53.204405 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 316/617] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:08:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 421 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:53.205977 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44466787-e654-4b86-a2fd-b521eaf34307 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:53.233789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4620e-688b-42c2-b2f4-1c518d6db2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:53.234786 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4620e-688b-42c2-b2f4-1c518d6db2ad None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:53.235224 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4620e-688b-42c2-b2f4-1c518d6db2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:53.235680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4620e-688b-42c2-b2f4-1c518d6db2ad None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:53.236737 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c4620e-688b-42c2-b2f4-1c518d6db2ad None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:53.237203 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c4620e-688b-42c2-b2f4-1c518d6db2ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:53.238816 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c4620e-688b-42c2-b2f4-1c518d6db2ad None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:53.250366 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 302/618] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Jul 25 19:08:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:53.251954 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4620e-688b-42c2-b2f4-1c518d6db2ad None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:53.378966 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4620e-688b-42c2-b2f4-1c518d6db2ad None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:53.421766 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44466787-e654-4b86-a2fd-b521eaf34307 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:53.425308 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06aef6-a94f-4b62-b068-1031220d308f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:53.426249 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06aef6-a94f-4b62-b068-1031220d308f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:53.426693 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06aef6-a94f-4b62-b068-1031220d308f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:53.427121 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06aef6-a94f-4b62-b068-1031220d308f None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:53.428073 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06aef6-a94f-4b62-b068-1031220d308f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:53.428803 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06aef6-a94f-4b62-b068-1031220d308f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:53.430384 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06aef6-a94f-4b62-b068-1031220d308f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:53.446879 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 303/619] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:08:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:53.448099 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06aef6-a94f-4b62-b068-1031220d308f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:53.471341 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:53.501877 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16857e9d-4766-4e20-9743-373b4ec75723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:53.503609 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16857e9d-4766-4e20-9743-373b4ec75723 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:53.503774 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16857e9d-4766-4e20-9743-373b4ec75723 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:53.504276 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16857e9d-4766-4e20-9743-373b4ec75723 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:53.524606 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16857e9d-4766-4e20-9743-373b4ec75723 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:08:53.540348 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-16857e9d-4766-4e20-9743-373b4ec75723 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:08:53.540348 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:53.540348 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:53.540348 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:53.540348 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:53.540348 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:53.540348 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:53.540348 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:53.540348 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:53.540348 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:53.540348 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:53.540348 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:53.540348 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:08:53.540348 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:08:53.540348 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:08:53.540348 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:08:53.541928 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:08:53.541928 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:08:53.541928 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:08:53.541928 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:08:53.541928 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:53.541928 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:53.541928 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:53.541928 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:53.541928 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:53.541928 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:53.541928 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:53.541928 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:53.541928 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:53.541928 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:53.541928 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:53.541928 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:53.543610 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:53.543610 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:53.543610 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:53.543610 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:53.543610 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:08:53.543610 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:08:53.543610 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:08:53.543610 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:08:53.543610 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:08:53.543610 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:08:53.543610 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:08:53.543610 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:53.543610 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 317/620] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Jul 25 19:08:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:53.544483 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16857e9d-4766-4e20-9743-373b4ec75723 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:53.595224 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06aef6-a94f-4b62-b068-1031220d308f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:53.641304 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86762f3f-07a6-4e56-a4d1-dec58e1011c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:53.642501 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86762f3f-07a6-4e56-a4d1-dec58e1011c9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:53.643001 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86762f3f-07a6-4e56-a4d1-dec58e1011c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:53.643432 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86762f3f-07a6-4e56-a4d1-dec58e1011c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:53.656931 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86762f3f-07a6-4e56-a4d1-dec58e1011c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:53.657254 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:53.659796 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86762f3f-07a6-4e56-a4d1-dec58e1011c9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:53.684521 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-86762f3f-07a6-4e56-a4d1-dec58e1011c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:53.684521 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:53.684521 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:53.684521 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:53.684521 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:53.684521 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:53.684521 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:53.684521 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:53.684521 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:53.684521 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:53.684521 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:53.684521 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:53.684521 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:08:53.684521 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:08:53.684521 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:08:53.685938 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:08:53.685938 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:53.685938 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:53.685938 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:53.685938 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:53.685938 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:53.685938 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:53.685938 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:53.685938 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:53.685938 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:53.685938 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:53.685938 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:53.685938 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:53.685938 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:53.685938 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:53.685938 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:53.685938 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:53.687062 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:08:53.687062 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:08:53.687062 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:53.687062 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:53.689618 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 304/621] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:08:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:53.691324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86762f3f-07a6-4e56-a4d1-dec58e1011c9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:53.756098 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16857e9d-4766-4e20-9743-373b4ec75723 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:53.800404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a3057-0ce3-4ab4-9d35-b7381a8f9541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:53.801005 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a3057-0ce3-4ab4-9d35-b7381a8f9541 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:53.801394 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a3057-0ce3-4ab4-9d35-b7381a8f9541 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:53.801772 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a3057-0ce3-4ab4-9d35-b7381a8f9541 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:53.812804 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a3057-0ce3-4ab4-9d35-b7381a8f9541 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:53.814707 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a3057-0ce3-4ab4-9d35-b7381a8f9541 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:53.817554 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:53.835729 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b49a3057-0ce3-4ab4-9d35-b7381a8f9541 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 19:08:53.835729 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:53.835729 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:53.835729 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:53.835729 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:53.835729 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:53.835729 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:53.835729 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:53.835729 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:53.835729 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:53.835729 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:53.835729 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:53.835729 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:08:53.835729 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:08:53.835729 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:08:53.835729 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:08:53.836830 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:53.836830 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:53.836830 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:08:53.836830 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:08:53.836830 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:08:53.836830 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:08:53.836830 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:53.836830 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:53.836830 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:53.836830 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:53.836830 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:53.836830 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:53.836830 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:53.836830 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:53.836830 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:53.836830 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:53.836830 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:53.836830 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:53.837794 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:53.837794 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:53.837794 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:08:53.837794 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:08:53.837794 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:08:53.837794 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:08:53.837794 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:08:53.837794 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:08:53.837794 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 19:08:53.837794 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:53.841851 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 318/622] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Jul 25 19:08:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:53.843541 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a3057-0ce3-4ab4-9d35-b7381a8f9541 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:53.890472 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86762f3f-07a6-4e56-a4d1-dec58e1011c9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:53.964288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c2e0be-8c10-4cf9-97bb-32f3f4cbb93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:53.966852 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c2e0be-8c10-4cf9-97bb-32f3f4cbb93d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:53.967048 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c2e0be-8c10-4cf9-97bb-32f3f4cbb93d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:53.967435 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c2e0be-8c10-4cf9-97bb-32f3f4cbb93d None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:53.968983 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c2e0be-8c10-4cf9-97bb-32f3f4cbb93d None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:53.969491 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c2e0be-8c10-4cf9-97bb-32f3f4cbb93d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:53.971992 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c2e0be-8c10-4cf9-97bb-32f3f4cbb93d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:53.984797 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a3057-0ce3-4ab4-9d35-b7381a8f9541 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:53.994441 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 305/623] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Jul 25 19:08:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:53.994441 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c2e0be-8c10-4cf9-97bb-32f3f4cbb93d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:54.009559 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:54.041912 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5308a310-d687-47df-92b4-15b28afe2428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:54.043246 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308a310-d687-47df-92b4-15b28afe2428 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:54.043814 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308a310-d687-47df-92b4-15b28afe2428 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:54.044317 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308a310-d687-47df-92b4-15b28afe2428 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:54.045437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5308a310-d687-47df-92b4-15b28afe2428 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:54.045976 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5308a310-d687-47df-92b4-15b28afe2428 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:54.048119 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5308a310-d687-47df-92b4-15b28afe2428 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:54.066176 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 319/624] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:08:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:54.067917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5308a310-d687-47df-92b4-15b28afe2428 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:54.211171 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c2e0be-8c10-4cf9-97bb-32f3f4cbb93d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:54.241423 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5308a310-d687-47df-92b4-15b28afe2428 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:54.261454 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:54.271324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac937c50-d4c5-42e6-b0ab-4e1e42e69e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:54.272534 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac937c50-d4c5-42e6-b0ab-4e1e42e69e07 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:54.273052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac937c50-d4c5-42e6-b0ab-4e1e42e69e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:54.273547 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac937c50-d4c5-42e6-b0ab-4e1e42e69e07 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:54.274663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac937c50-d4c5-42e6-b0ab-4e1e42e69e07 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:54.275532 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac937c50-d4c5-42e6-b0ab-4e1e42e69e07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:54.277356 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac937c50-d4c5-42e6-b0ab-4e1e42e69e07 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:54.309659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 306/625] 10.222.0.18 () {56 vars in 1135 bytes} [Mon Jul 25 19:08:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:54.312074 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac937c50-d4c5-42e6-b0ab-4e1e42e69e07 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:54.317510 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08730463-0916-4104-9814-90ce39f9f161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:54.318919 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08730463-0916-4104-9814-90ce39f9f161 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:54.319428 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08730463-0916-4104-9814-90ce39f9f161 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:54.319968 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08730463-0916-4104-9814-90ce39f9f161 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:54.334908 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08730463-0916-4104-9814-90ce39f9f161 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:54.337151 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08730463-0916-4104-9814-90ce39f9f161 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:54.362333 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-08730463-0916-4104-9814-90ce39f9f161 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:54.362333 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:54.362333 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:54.362333 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:54.362333 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:54.362333 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:54.362333 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:54.362333 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:54.362333 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:54.362333 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:54.362333 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:54.362333 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:54.362333 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:08:54.362333 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:08:54.362333 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:08:54.363404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:08:54.363404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:54.363404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:54.363404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:54.363404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:54.363404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:54.363404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:54.363404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:54.363404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:54.363404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:54.363404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:54.363404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:54.363404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:54.363404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:54.363404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:54.363404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:54.363404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:54.364481 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:08:54.364481 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:08:54.364481 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:54.364481 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:54.371453 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 320/626] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:08:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:54.372841 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08730463-0916-4104-9814-90ce39f9f161 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:54.412069 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:54.539812 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08730463-0916-4104-9814-90ce39f9f161 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:54.547306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac937c50-d4c5-42e6-b0ab-4e1e42e69e07 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:54.601163 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe13fbc-e1ec-4065-8dd3-56c3733258cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:54.604037 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe13fbc-e1ec-4065-8dd3-56c3733258cd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:54.605180 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe13fbc-e1ec-4065-8dd3-56c3733258cd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:54.606605 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe13fbc-e1ec-4065-8dd3-56c3733258cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:54.611572 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4e83b-899a-4f5d-a140-807d74703b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:54.612387 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4e83b-899a-4f5d-a140-807d74703b76 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:54.612740 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4e83b-899a-4f5d-a140-807d74703b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:54.612988 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4e83b-899a-4f5d-a140-807d74703b76 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:54.622613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f4e83b-899a-4f5d-a140-807d74703b76 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:54.624317 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f4e83b-899a-4f5d-a140-807d74703b76 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:54.634537 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe13fbc-e1ec-4065-8dd3-56c3733258cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:54.639124 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe13fbc-e1ec-4065-8dd3-56c3733258cd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:54.645319 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-c0f4e83b-899a-4f5d-a140-807d74703b76 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 19:08:54.645319 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:54.645319 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:54.645319 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:54.645319 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:54.645319 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:54.645319 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:54.645319 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:54.645319 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:54.645319 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:54.645319 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:54.645319 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:54.645319 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:08:54.645319 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:08:54.645319 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:08:54.645319 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:08:54.647172 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:54.647172 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:54.647172 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:08:54.647172 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:08:54.647172 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:08:54.647172 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:08:54.647172 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:54.647172 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:54.647172 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:54.647172 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:54.647172 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:54.647172 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:54.647172 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:54.647172 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:54.647172 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:54.647172 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:54.647172 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:54.647172 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:54.650191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:54.650191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:54.650191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:08:54.650191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:08:54.650191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:08:54.650191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:08:54.650191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:08:54.650191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:08:54.650191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 19:08:54.650191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:54.650191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 307/627] 10.222.0.18 () {56 vars in 1093 bytes} [Mon Jul 25 19:08:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:54.650191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4e83b-899a-4f5d-a140-807d74703b76 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:54.686668 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-bbe13fbc-e1ec-4065-8dd3-56c3733258cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:54.686668 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:54.686668 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:54.686668 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:54.686668 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:54.686668 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:54.686668 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:54.686668 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:54.686668 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:54.686668 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:54.686668 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:54.686668 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:54.686668 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:08:54.686668 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:08:54.686668 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:08:54.688377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:08:54.688377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:54.688377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:54.688377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:54.688377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:54.688377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:54.688377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:54.688377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:54.688377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:54.688377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:54.688377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:54.688377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:54.688377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:54.688377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:54.688377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:54.688377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:54.688377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:54.690121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:08:54.690121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:08:54.690121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:54.690121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:54.690121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 321/628] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:08:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:54.690121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe13fbc-e1ec-4065-8dd3-56c3733258cd None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:54.754507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:54.834209 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:54.841981 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4e83b-899a-4f5d-a140-807d74703b76 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:54.890871 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb47bd2-6424-433a-8738-6158f73e0393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:54.892058 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb47bd2-6424-433a-8738-6158f73e0393 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:54.892554 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb47bd2-6424-433a-8738-6158f73e0393 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:54.893136 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb47bd2-6424-433a-8738-6158f73e0393 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:54.894228 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb47bd2-6424-433a-8738-6158f73e0393 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:54.894768 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb47bd2-6424-433a-8738-6158f73e0393 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:54.896090 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe13fbc-e1ec-4065-8dd3-56c3733258cd None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:54.896812 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb47bd2-6424-433a-8738-6158f73e0393 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:54.918388 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 308/629] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:08:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:54.918388 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb47bd2-6424-433a-8738-6158f73e0393 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:54.975936 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf97fb3a-8563-41fe-b567-c5f5c23b4092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:54.976648 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97fb3a-8563-41fe-b567-c5f5c23b4092 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:54.976937 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97fb3a-8563-41fe-b567-c5f5c23b4092 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:54.977228 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97fb3a-8563-41fe-b567-c5f5c23b4092 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:54.978144 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf97fb3a-8563-41fe-b567-c5f5c23b4092 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:54.978476 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf97fb3a-8563-41fe-b567-c5f5c23b4092 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:54.981543 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf97fb3a-8563-41fe-b567-c5f5c23b4092 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:55.004796 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 322/630] 10.222.0.18 () {56 vars in 1133 bytes} [Mon Jul 25 19:08:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:55.006190 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbcbd5-97ad-4e54-96d2-e3665b3c57cf None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:55.006480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbcbd5-97ad-4e54-96d2-e3665b3c57cf None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:55.006760 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbcbd5-97ad-4e54-96d2-e3665b3c57cf None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:55.008511 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 323/631] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:08:55.009331 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cbcbd5-97ad-4e54-96d2-e3665b3c57cf None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:55.103903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb47bd2-6424-433a-8738-6158f73e0393 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:55.117880 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:55.158979 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb034675-d9d4-4dbe-b24a-74122abe2987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:55.159745 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cbcbd5-97ad-4e54-96d2-e3665b3c57cf None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:55.160471 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb034675-d9d4-4dbe-b24a-74122abe2987 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:55.160810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb034675-d9d4-4dbe-b24a-74122abe2987 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:55.161331 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb034675-d9d4-4dbe-b24a-74122abe2987 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:55.162668 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb034675-d9d4-4dbe-b24a-74122abe2987 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:55.163230 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb034675-d9d4-4dbe-b24a-74122abe2987 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:55.165614 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb034675-d9d4-4dbe-b24a-74122abe2987 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:55.203589 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 309/632] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:08:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:55.203589 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb034675-d9d4-4dbe-b24a-74122abe2987 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:55.225675 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ab878-e2bc-4c9f-91bb-efb303b47271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:55.226821 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ab878-e2bc-4c9f-91bb-efb303b47271 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:55.227247 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ab878-e2bc-4c9f-91bb-efb303b47271 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:55.227717 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ab878-e2bc-4c9f-91bb-efb303b47271 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:55.231898 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:55.245384 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39ab878-e2bc-4c9f-91bb-efb303b47271 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:55.248984 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39ab878-e2bc-4c9f-91bb-efb303b47271 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:55.274072 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d39ab878-e2bc-4c9f-91bb-efb303b47271 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 25 19:08:55.274072 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:55.274072 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:55.274072 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:55.274072 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:55.274072 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:55.274072 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:55.274072 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:55.274072 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:55.274072 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:55.274072 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:55.274072 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:55.274072 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:08:55.274072 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:08:55.274072 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:08:55.274072 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:08:55.285869 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:55.285869 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:55.285869 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:08:55.285869 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:08:55.285869 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:08:55.285869 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:08:55.285869 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:55.285869 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:55.285869 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:55.285869 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:55.285869 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:55.285869 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:55.285869 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:55.285869 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:55.285869 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:55.285869 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:55.285869 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:55.285869 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:55.288257 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:55.288257 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:55.288257 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:08:55.288257 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:08:55.288257 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:08:55.288257 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:08:55.288257 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:08:55.288257 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:08:55.288257 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 25 19:08:55.288257 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:55.288257 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 324/633] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:08:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:55.288257 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139f79e-e44b-4d46-a75b-20fdfe58fd30 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:55.288257 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139f79e-e44b-4d46-a75b-20fdfe58fd30 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:55.289771 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139f79e-e44b-4d46-a75b-20fdfe58fd30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:55.300243 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c139f79e-e44b-4d46-a75b-20fdfe58fd30 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:55.301492 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c139f79e-e44b-4d46-a75b-20fdfe58fd30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:55.390560 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb034675-d9d4-4dbe-b24a-74122abe2987 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:55.391717 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 325/634] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:08:55.392693 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c139f79e-e44b-4d46-a75b-20fdfe58fd30 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:55.477399 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1205bb8-3613-48a0-8efd-5af6cc0383d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:55.478808 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1205bb8-3613-48a0-8efd-5af6cc0383d5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:55.479473 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1205bb8-3613-48a0-8efd-5af6cc0383d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:55.480205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1205bb8-3613-48a0-8efd-5af6cc0383d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:55.550470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:55.571339 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c139f79e-e44b-4d46-a75b-20fdfe58fd30 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:55.613704 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1205bb8-3613-48a0-8efd-5af6cc0383d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:55.616058 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1205bb8-3613-48a0-8efd-5af6cc0383d5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:55.616613 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:55.637703 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f656d72-d3bb-487f-bc22-5b48eff860e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:55.638699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f656d72-d3bb-487f-bc22-5b48eff860e3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:55.639158 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f656d72-d3bb-487f-bc22-5b48eff860e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:55.639643 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f656d72-d3bb-487f-bc22-5b48eff860e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:55.651124 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f656d72-d3bb-487f-bc22-5b48eff860e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:55.654985 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f656d72-d3bb-487f-bc22-5b48eff860e3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:55.674997 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-5f656d72-d3bb-487f-bc22-5b48eff860e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:08:55.674997 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:55.674997 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:55.674997 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:55.674997 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:55.674997 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:55.674997 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:55.674997 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:55.674997 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:55.674997 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:55.674997 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:55.674997 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:55.674997 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:08:55.674997 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:08:55.674997 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:08:55.676310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:08:55.676310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:55.676310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:55.676310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:55.676310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:55.676310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:55.676310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:55.676310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:55.676310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:55.676310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:55.676310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:55.676310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:55.676310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:55.676310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:55.676310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:55.676310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:55.676310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:55.677276 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:08:55.677276 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:08:55.677276 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:08:55.677276 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:08:55.677276 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:08:55.677276 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:08:55.677276 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:08:55.677276 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:55.685630 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 326/635] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:08:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:55.686949 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f656d72-d3bb-487f-bc22-5b48eff860e3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:55.791497 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 310/636] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:08:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 19:08:55.793571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1205bb8-3613-48a0-8efd-5af6cc0383d5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:55.835000 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f656d72-d3bb-487f-bc22-5b48eff860e3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:55.875348 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7920bbb6-222e-4c58-9183-d0d107da97f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:55.876309 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920bbb6-222e-4c58-9183-d0d107da97f9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:55.876751 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920bbb6-222e-4c58-9183-d0d107da97f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:55.877213 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920bbb6-222e-4c58-9183-d0d107da97f9 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:55.878349 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7920bbb6-222e-4c58-9183-d0d107da97f9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:55.878714 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7920bbb6-222e-4c58-9183-d0d107da97f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:55.880501 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7920bbb6-222e-4c58-9183-d0d107da97f9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:55.885796 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:55.905154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 327/637] 10.222.0.18 () {56 vars in 1137 bytes} [Mon Jul 25 19:08:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:55.906722 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7920bbb6-222e-4c58-9183-d0d107da97f9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:55.937102 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1205bb8-3613-48a0-8efd-5af6cc0383d5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:55.967901 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:55.986053 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe6d368-8f54-4739-be25-2dc688b0fb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:55.987067 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6d368-8f54-4739-be25-2dc688b0fb37 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:55.987542 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6d368-8f54-4739-be25-2dc688b0fb37 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:55.987906 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6d368-8f54-4739-be25-2dc688b0fb37 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:56.004252 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe6d368-8f54-4739-be25-2dc688b0fb37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:08:56.014324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-4fe6d368-8f54-4739-be25-2dc688b0fb37 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:08:56.014324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:56.014324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:56.014324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:56.014324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:56.014324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:56.014324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:56.014324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:56.014324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:56.014324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:56.014324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:56.014324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:56.014324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:08:56.014324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:08:56.014324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:08:56.014324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:08:56.016423 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:08:56.016423 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:08:56.016423 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:08:56.016423 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:08:56.016423 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:56.016423 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:56.016423 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:56.016423 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:56.016423 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:56.016423 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:56.016423 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:56.016423 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:56.016423 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:56.016423 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:56.016423 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:56.016423 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:56.018029 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:56.018029 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:56.018029 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:56.018029 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:56.018029 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:08:56.018029 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:08:56.018029 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:08:56.018029 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:08:56.018029 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:08:56.018029 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:08:56.018029 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:08:56.018029 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:56.019846 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 311/638] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Jul 25 19:08:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:56.022564 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4916b7-ae4a-4b9e-9b73-f342effd78e3 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:56.023067 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4916b7-ae4a-4b9e-9b73-f342effd78e3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:56.023600 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4916b7-ae4a-4b9e-9b73-f342effd78e3 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:56.025646 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 312/639] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:08:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:08:56.026725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4916b7-ae4a-4b9e-9b73-f342effd78e3 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:56.060125 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7920bbb6-222e-4c58-9183-d0d107da97f9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:56.109228 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0186447-6a51-47b2-8fc1-a3cd7679b3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:56.110410 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0186447-6a51-47b2-8fc1-a3cd7679b3d2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:56.111028 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0186447-6a51-47b2-8fc1-a3cd7679b3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:56.111411 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0186447-6a51-47b2-8fc1-a3cd7679b3d2 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:56.112961 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0186447-6a51-47b2-8fc1-a3cd7679b3d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:56.113524 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0186447-6a51-47b2-8fc1-a3cd7679b3d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:56.116185 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0186447-6a51-47b2-8fc1-a3cd7679b3d2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:56.138302 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 328/640] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Jul 25 19:08:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:56.139988 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0186447-6a51-47b2-8fc1-a3cd7679b3d2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:56.184820 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4916b7-ae4a-4b9e-9b73-f342effd78e3 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:56.237636 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:56.251301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236fffd4-c089-4539-83d3-4246f9d93af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:56.252676 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236fffd4-c089-4539-83d3-4246f9d93af3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:56.253180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236fffd4-c089-4539-83d3-4246f9d93af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:56.253597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236fffd4-c089-4539-83d3-4246f9d93af3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:56.270465 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236fffd4-c089-4539-83d3-4246f9d93af3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:56.272261 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236fffd4-c089-4539-83d3-4246f9d93af3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:56.284950 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0186447-6a51-47b2-8fc1-a3cd7679b3d2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:56.286927 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:56.295976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-236fffd4-c089-4539-83d3-4246f9d93af3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:56.295976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:56.295976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:56.295976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:56.295976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:56.295976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:56.295976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:56.295976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:56.295976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:56.295976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:56.295976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:56.295976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:56.295976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:08:56.295976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:08:56.295976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:08:56.298063 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:08:56.298063 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:56.298063 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:56.298063 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:56.298063 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:56.298063 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:56.298063 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:56.298063 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:56.298063 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:56.298063 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:56.298063 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:56.298063 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:56.298063 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:56.298063 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:56.298063 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:56.298063 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:56.298063 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:56.300138 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:08:56.300138 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:08:56.300138 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:56.300138 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:56.300138 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 313/641] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:08:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:56.300138 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236fffd4-c089-4539-83d3-4246f9d93af3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:56.337358 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725638a-8a37-4522-bfd5-12bf74d27a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:56.338033 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725638a-8a37-4522-bfd5-12bf74d27a78 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:56.338320 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725638a-8a37-4522-bfd5-12bf74d27a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:56.338596 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725638a-8a37-4522-bfd5-12bf74d27a78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:56.432880 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725638a-8a37-4522-bfd5-12bf74d27a78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:56.436099 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725638a-8a37-4522-bfd5-12bf74d27a78 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:56.452016 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236fffd4-c089-4539-83d3-4246f9d93af3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:56.496436 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa76fe5-6cb5-4f9c-9d07-c3229fe22a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:56.497280 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa76fe5-6cb5-4f9c-9d07-c3229fe22a84 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:56.497596 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa76fe5-6cb5-4f9c-9d07-c3229fe22a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:56.497867 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa76fe5-6cb5-4f9c-9d07-c3229fe22a84 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:56.498655 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa76fe5-6cb5-4f9c-9d07-c3229fe22a84 None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:56.498966 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa76fe5-6cb5-4f9c-9d07-c3229fe22a84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:56.500437 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa76fe5-6cb5-4f9c-9d07-c3229fe22a84 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:56.510310 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 314/642] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Jul 25 19:08:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:56.513576 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896dc4d7-ecc6-45a1-b558-f88cb4c2988e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:56.514027 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896dc4d7-ecc6-45a1-b558-f88cb4c2988e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:56.514454 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896dc4d7-ecc6-45a1-b558-f88cb4c2988e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:56.538258 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-896dc4d7-ecc6-45a1-b558-f88cb4c2988e None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:08:56.539836 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-896dc4d7-ecc6-45a1-b558-f88cb4c2988e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:08:56.588054 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 329/643] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:08:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 19:08:56.588900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725638a-8a37-4522-bfd5-12bf74d27a78 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:56.593763 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:56.606990 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:56.648640 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 315/644] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:08:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:08:56.650240 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896dc4d7-ecc6-45a1-b558-f88cb4c2988e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:56.755091 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725638a-8a37-4522-bfd5-12bf74d27a78 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:56.800468 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af8426-3527-4e84-993c-2801ff7b7fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:56.801480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af8426-3527-4e84-993c-2801ff7b7fd0 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:56.801923 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af8426-3527-4e84-993c-2801ff7b7fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:56.802258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af8426-3527-4e84-993c-2801ff7b7fd0 None admin] PATH_INFO: `/v3/projects/191069caccd04233b012585e29cfa18b/users/feafcf1b5a9844fe9738823fddd07e0e/roles/23718d83b0864b16b0071e8ffe0b5861` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:56.806501 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896dc4d7-ecc6-45a1-b558-f88cb4c2988e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:56.821759 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af8426-3527-4e84-993c-2801ff7b7fd0 None admin] RBAC: Authorizing `identity:create_grant(project_id=191069caccd04233b012585e29cfa18b, user_id=feafcf1b5a9844fe9738823fddd07e0e, role_id=23718d83b0864b16b0071e8ffe0b5861)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:56.822453 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af8426-3527-4e84-993c-2801ff7b7fd0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:56.844902 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 330/645] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Jul 25 19:08:56 2022] PUT /identity/v3/projects/191069caccd04233b012585e29cfa18b/users/feafcf1b5a9844fe9738823fddd07e0e/roles/23718d83b0864b16b0071e8ffe0b5861 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:08:56.846808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af8426-3527-4e84-993c-2801ff7b7fd0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:56.861166 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee18e2b-cedc-4ceb-ace6-94c66242b0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:56.862143 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee18e2b-cedc-4ceb-ace6-94c66242b0c5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:56.862552 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee18e2b-cedc-4ceb-ace6-94c66242b0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:56.862977 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee18e2b-cedc-4ceb-ace6-94c66242b0c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:56.864144 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee18e2b-cedc-4ceb-ace6-94c66242b0c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:56.864557 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee18e2b-cedc-4ceb-ace6-94c66242b0c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:56.865952 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee18e2b-cedc-4ceb-ace6-94c66242b0c5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:56.882725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 316/646] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:08:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:56.883474 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee18e2b-cedc-4ceb-ace6-94c66242b0c5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:56.927066 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:56.973527 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:57.058022 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee18e2b-cedc-4ceb-ace6-94c66242b0c5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:57.082945 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af8426-3527-4e84-993c-2801ff7b7fd0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:57.122268 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d69f62-6f33-4db4-8822-2f749d9e6d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:57.123569 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d69f62-6f33-4db4-8822-2f749d9e6d3c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:57.123879 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d69f62-6f33-4db4-8822-2f749d9e6d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:57.124165 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d69f62-6f33-4db4-8822-2f749d9e6d3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:57.139192 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d69f62-6f33-4db4-8822-2f749d9e6d3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:57.141635 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d69f62-6f33-4db4-8822-2f749d9e6d3c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:57.159252 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddfda65-03ab-426c-a8c6-a501efde92a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:57.160725 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddfda65-03ab-426c-a8c6-a501efde92a4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:57.161222 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddfda65-03ab-426c-a8c6-a501efde92a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:57.161872 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddfda65-03ab-426c-a8c6-a501efde92a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:57.166012 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-05d69f62-6f33-4db4-8822-2f749d9e6d3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:57.166012 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:57.166012 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:57.166012 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:57.166012 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:57.166012 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:57.166012 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:57.166012 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:57.166012 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:57.166012 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:57.166012 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:57.166012 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:57.166012 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:08:57.166012 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:08:57.166012 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:08:57.168014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:08:57.168014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:57.168014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:57.168014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:57.168014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:57.168014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:57.168014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:57.168014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:57.168014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:57.168014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:57.168014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:57.168014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:57.168014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:57.168014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:57.168014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:57.168014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:57.168014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:57.169611 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:08:57.169611 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:08:57.169611 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:57.169611 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:57.172983 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 317/647] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:08:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:57.173788 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d69f62-6f33-4db4-8822-2f749d9e6d3c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:57.181846 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddfda65-03ab-426c-a8c6-a501efde92a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:57.183975 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddfda65-03ab-426c-a8c6-a501efde92a4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:57.208644 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-3ddfda65-03ab-426c-a8c6-a501efde92a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:57.208644 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:57.208644 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:57.208644 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:57.208644 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:57.208644 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:57.208644 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:57.208644 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:57.208644 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:57.208644 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:57.208644 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:57.208644 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:57.208644 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:08:57.208644 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:08:57.208644 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:08:57.209757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:08:57.209757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:57.209757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:57.209757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:57.209757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:57.209757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:57.209757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:57.209757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:57.209757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:57.209757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:57.209757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:57.209757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:57.209757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:57.209757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:57.209757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:57.209757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:57.209757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:57.210888 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:08:57.210888 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:08:57.210888 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:57.210888 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:57.211135 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 331/648] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:08:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:57.212132 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddfda65-03ab-426c-a8c6-a501efde92a4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:57.286827 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:57.311002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d69f62-6f33-4db4-8822-2f749d9e6d3c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:57.364456 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69aade-c115-44cf-af96-cae1a2ecf52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:57.365953 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69aade-c115-44cf-af96-cae1a2ecf52c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:57.366240 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69aade-c115-44cf-af96-cae1a2ecf52c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:57.366520 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69aade-c115-44cf-af96-cae1a2ecf52c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:57.381627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b69aade-c115-44cf-af96-cae1a2ecf52c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:08:57.391120 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-1b69aade-c115-44cf-af96-cae1a2ecf52c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:08:57.391120 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:57.391120 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:57.391120 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:57.391120 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:57.391120 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:57.391120 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:57.391120 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:57.391120 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:57.391120 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:57.391120 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:57.391120 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:57.391120 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:08:57.391120 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:08:57.391120 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:08:57.391120 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:08:57.392338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:08:57.392338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:08:57.392338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:08:57.392338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:08:57.392338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:57.392338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:57.392338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:57.392338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:57.392338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:57.392338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:57.392338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:57.392338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:57.392338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:57.392338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:57.392338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:57.392338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:57.393307 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:57.393307 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:57.393307 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:57.393307 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:57.393307 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:08:57.393307 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:08:57.393307 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:08:57.393307 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:08:57.393307 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:08:57.393307 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:08:57.393307 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:08:57.393307 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:57.393307 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 318/649] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Jul 25 19:08:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:57.393307 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69aade-c115-44cf-af96-cae1a2ecf52c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:57.420090 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddfda65-03ab-426c-a8c6-a501efde92a4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:57.424968 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:57.487287 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae0685-6f58-4e87-acdd-a4e1025fcaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:57.488902 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae0685-6f58-4e87-acdd-a4e1025fcaf6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:57.489351 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae0685-6f58-4e87-acdd-a4e1025fcaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:57.489852 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae0685-6f58-4e87-acdd-a4e1025fcaf6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:57.503181 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ae0685-6f58-4e87-acdd-a4e1025fcaf6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:57.505790 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ae0685-6f58-4e87-acdd-a4e1025fcaf6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:57.532756 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-a2ae0685-6f58-4e87-acdd-a4e1025fcaf6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:08:57.532756 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:57.532756 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:57.532756 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:57.532756 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:57.532756 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:57.532756 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:57.532756 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:57.532756 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:57.532756 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:57.532756 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:57.532756 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:57.532756 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:08:57.532756 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:08:57.532756 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:08:57.533732 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:08:57.533732 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:57.533732 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:57.533732 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:57.533732 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:57.533732 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:57.533732 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:57.533732 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:57.533732 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:57.533732 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:57.533732 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:57.533732 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:57.533732 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:57.533732 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:57.533732 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:57.533732 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:57.533732 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:57.534658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:08:57.534658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:08:57.534658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:08:57.534658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:08:57.534658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:08:57.534658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:08:57.534658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:08:57.534658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:57.535101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 332/650] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:08:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:57.536621 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae0685-6f58-4e87-acdd-a4e1025fcaf6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:57.564117 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69aade-c115-44cf-af96-cae1a2ecf52c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:57.611092 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7d1acc-ad18-409b-8717-5846a03719a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:57.611815 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d1acc-ad18-409b-8717-5846a03719a6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:57.612100 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d1acc-ad18-409b-8717-5846a03719a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:57.612375 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d1acc-ad18-409b-8717-5846a03719a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:57.613186 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7d1acc-ad18-409b-8717-5846a03719a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:57.613540 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7d1acc-ad18-409b-8717-5846a03719a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:57.614974 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7d1acc-ad18-409b-8717-5846a03719a6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:57.631273 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 319/651] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:08:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:57.633308 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7d1acc-ad18-409b-8717-5846a03719a6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:57.641441 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:57.708657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae0685-6f58-4e87-acdd-a4e1025fcaf6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:57.780138 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c358b8-13de-4d77-bffc-a09f444c48e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:57.781655 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c358b8-13de-4d77-bffc-a09f444c48e8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:57.782132 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c358b8-13de-4d77-bffc-a09f444c48e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:57.782672 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c358b8-13de-4d77-bffc-a09f444c48e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:57.785271 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7d1acc-ad18-409b-8717-5846a03719a6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:57.794015 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c358b8-13de-4d77-bffc-a09f444c48e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:57.794673 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c358b8-13de-4d77-bffc-a09f444c48e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:57.798827 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c358b8-13de-4d77-bffc-a09f444c48e8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:57.822948 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 333/652] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:08:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:57.825128 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c358b8-13de-4d77-bffc-a09f444c48e8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:57.842810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4acfdd3-4176-48f6-befe-801caa1e86dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:57.843980 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4acfdd3-4176-48f6-befe-801caa1e86dd None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:57.844456 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4acfdd3-4176-48f6-befe-801caa1e86dd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:57.844927 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4acfdd3-4176-48f6-befe-801caa1e86dd None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:57.846300 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4acfdd3-4176-48f6-befe-801caa1e86dd None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:57.846826 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4acfdd3-4176-48f6-befe-801caa1e86dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:57.849424 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4acfdd3-4176-48f6-befe-801caa1e86dd None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:57.861470 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:57.870373 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 320/653] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Jul 25 19:08:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:57.870373 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4acfdd3-4176-48f6-befe-801caa1e86dd None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:57.969169 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c358b8-13de-4d77-bffc-a09f444c48e8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:57.974522 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:58.024165 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8285b651-6490-4130-bbe3-cf22d847370b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:58.024505 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4acfdd3-4176-48f6-befe-801caa1e86dd None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:58.025106 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285b651-6490-4130-bbe3-cf22d847370b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:58.025520 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285b651-6490-4130-bbe3-cf22d847370b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:58.025942 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285b651-6490-4130-bbe3-cf22d847370b None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:58.027673 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8285b651-6490-4130-bbe3-cf22d847370b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:58.028198 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8285b651-6490-4130-bbe3-cf22d847370b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:58.030796 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8285b651-6490-4130-bbe3-cf22d847370b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:58.048562 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 334/654] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Jul 25 19:08:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:58.049616 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8285b651-6490-4130-bbe3-cf22d847370b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:58.089487 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5273eb-cb64-4666-9e9e-e3eeae4e7cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:58.090997 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5273eb-cb64-4666-9e9e-e3eeae4e7cdc None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:58.091871 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5273eb-cb64-4666-9e9e-e3eeae4e7cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:58.092072 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5273eb-cb64-4666-9e9e-e3eeae4e7cdc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:58.101264 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5273eb-cb64-4666-9e9e-e3eeae4e7cdc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:58.102847 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5273eb-cb64-4666-9e9e-e3eeae4e7cdc None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:58.120126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-dd5273eb-cb64-4666-9e9e-e3eeae4e7cdc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:08:58.120126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:58.120126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:58.120126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:58.120126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:58.120126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:58.120126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:58.120126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:58.120126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:58.120126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:58.120126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:58.120126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:58.120126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:08:58.120126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:08:58.120126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:08:58.121365 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:08:58.121365 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:58.121365 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:58.121365 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:58.121365 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:58.121365 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:58.121365 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:58.121365 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:58.121365 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:58.121365 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:58.121365 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:58.121365 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:58.121365 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:58.121365 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:58.121365 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:58.121365 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:58.121365 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:58.124162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:08:58.124162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:08:58.124162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:08:58.124162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:08:58.124162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:08:58.124162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:08:58.124162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:08:58.124162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:58.124162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 321/655] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:08:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:58.124162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5273eb-cb64-4666-9e9e-e3eeae4e7cdc None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:58.204186 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:58.234524 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8285b651-6490-4130-bbe3-cf22d847370b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:58.299550 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5273eb-cb64-4666-9e9e-e3eeae4e7cdc None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:58.302370 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d02125-2c1e-42bc-be01-ab5a6ee4b07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:58.303804 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d02125-2c1e-42bc-be01-ab5a6ee4b07a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:58.304307 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d02125-2c1e-42bc-be01-ab5a6ee4b07a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:58.304790 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d02125-2c1e-42bc-be01-ab5a6ee4b07a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:58.319112 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d02125-2c1e-42bc-be01-ab5a6ee4b07a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:58.320962 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d02125-2c1e-42bc-be01-ab5a6ee4b07a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:58.343407 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:58.351087 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-89d02125-2c1e-42bc-be01-ab5a6ee4b07a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 19:08:58.351087 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:58.351087 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:58.351087 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:58.351087 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:58.351087 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:58.351087 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:58.351087 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:58.351087 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:58.351087 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:58.351087 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:58.351087 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:58.351087 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:08:58.351087 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:08:58.351087 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:08:58.351087 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:08:58.353165 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:58.353165 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:58.353165 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:08:58.353165 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:08:58.353165 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:08:58.353165 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:08:58.353165 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:58.353165 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:58.353165 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:58.353165 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:58.353165 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:58.353165 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:58.353165 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:58.353165 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:58.353165 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:58.353165 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:58.353165 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:58.353165 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:58.354996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:58.354996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:58.354996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:08:58.354996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:08:58.354996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:08:58.354996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:08:58.354996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:08:58.354996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:08:58.354996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 19:08:58.354996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:08:58.354996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 335/656] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:08:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:58.354996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d02125-2c1e-42bc-be01-ab5a6ee4b07a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:58.364992 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b865247-d4e8-417d-b131-ab036b6003b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:58.366326 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b865247-d4e8-417d-b131-ab036b6003b3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:58.366785 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b865247-d4e8-417d-b131-ab036b6003b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:58.367246 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b865247-d4e8-417d-b131-ab036b6003b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:58.384250 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b865247-d4e8-417d-b131-ab036b6003b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:58.386137 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b865247-d4e8-417d-b131-ab036b6003b3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:58.417147 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-9b865247-d4e8-417d-b131-ab036b6003b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:58.417147 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:58.417147 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:58.417147 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:58.417147 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:58.417147 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:58.417147 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:58.417147 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:58.417147 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:58.417147 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:58.417147 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:58.417147 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:58.417147 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:08:58.417147 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:08:58.417147 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:08:58.418810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:08:58.418810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:58.418810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:58.418810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:58.418810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:58.418810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:58.418810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:58.418810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:58.418810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:58.418810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:58.418810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:58.418810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:58.418810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:58.418810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:58.418810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:58.418810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:58.418810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:58.419903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:08:58.419903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:08:58.419903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:08:58.419903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:58.424093 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 322/657] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:08:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:58.425065 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b865247-d4e8-417d-b131-ab036b6003b3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:58.578744 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d02125-2c1e-42bc-be01-ab5a6ee4b07a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:58.590670 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b865247-d4e8-417d-b131-ab036b6003b3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:58.634645 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf213a0-1da3-49bb-83fb-a70fa3e390ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:58.635270 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf213a0-1da3-49bb-83fb-a70fa3e390ec None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:58.635584 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf213a0-1da3-49bb-83fb-a70fa3e390ec None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:58.635873 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf213a0-1da3-49bb-83fb-a70fa3e390ec None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:58.636974 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf213a0-1da3-49bb-83fb-a70fa3e390ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:58.637508 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf213a0-1da3-49bb-83fb-a70fa3e390ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:58.639178 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf213a0-1da3-49bb-83fb-a70fa3e390ec None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:58.651732 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b094864-4090-4dcf-9a03-369a5f79eab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:58.653836 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b094864-4090-4dcf-9a03-369a5f79eab5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:58.654296 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b094864-4090-4dcf-9a03-369a5f79eab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:58.654786 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b094864-4090-4dcf-9a03-369a5f79eab5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:58.660502 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 323/658] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Jul 25 19:08:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:58.660502 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf213a0-1da3-49bb-83fb-a70fa3e390ec None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:58.660797 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b094864-4090-4dcf-9a03-369a5f79eab5 None admin] RBAC: Adding query filter params (role.id=23718d83b0864b16b0071e8ffe0b5861, scope.project.id=191069caccd04233b012585e29cfa18b, user.id=860e7d26ad364795bb25836aadd4ac7b) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:58.660797 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b094864-4090-4dcf-9a03-369a5f79eab5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:58.660797 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b094864-4090-4dcf-9a03-369a5f79eab5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:58.672416 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 336/659] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:08:58 2022] GET /identity/v3/role_assignments?user.id=860e7d26ad364795bb25836aadd4ac7b&scope.project.id=191069caccd04233b012585e29cfa18b&role.id=23718d83b0864b16b0071e8ffe0b5861 => generated 608 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:58.675698 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b094864-4090-4dcf-9a03-369a5f79eab5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:58.702896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:58.750616 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:58.861189 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b094864-4090-4dcf-9a03-369a5f79eab5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:58.884066 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf213a0-1da3-49bb-83fb-a70fa3e390ec None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:58.917782 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb130a5-29ee-44b6-bfa9-c232b9b5e06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:58.918558 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb130a5-29ee-44b6-bfa9-c232b9b5e06d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:58.918862 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb130a5-29ee-44b6-bfa9-c232b9b5e06d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:58.919223 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb130a5-29ee-44b6-bfa9-c232b9b5e06d None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:58.920122 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb130a5-29ee-44b6-bfa9-c232b9b5e06d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:58.920512 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb130a5-29ee-44b6-bfa9-c232b9b5e06d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:58.922631 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb130a5-29ee-44b6-bfa9-c232b9b5e06d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:58.941199 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 337/660] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:08:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:58.942416 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb130a5-29ee-44b6-bfa9-c232b9b5e06d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:58.968680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a971322-7f85-4102-bfd0-36629bbb98b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:58.969196 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a971322-7f85-4102-bfd0-36629bbb98b6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:58.969648 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a971322-7f85-4102-bfd0-36629bbb98b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:58.970039 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a971322-7f85-4102-bfd0-36629bbb98b6 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:58.971051 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a971322-7f85-4102-bfd0-36629bbb98b6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:58.971482 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a971322-7f85-4102-bfd0-36629bbb98b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:58.973213 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a971322-7f85-4102-bfd0-36629bbb98b6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:59.002669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 324/661] 10.222.0.18 () {56 vars in 1137 bytes} [Mon Jul 25 19:08:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:59.004964 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a971322-7f85-4102-bfd0-36629bbb98b6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:59.075077 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:59.132875 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb130a5-29ee-44b6-bfa9-c232b9b5e06d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:59.172794 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:59.194233 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a971322-7f85-4102-bfd0-36629bbb98b6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:59.207582 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918418f-c22a-4e21-a269-e3eee0954d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:59.209045 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918418f-c22a-4e21-a269-e3eee0954d7e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:59.209647 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918418f-c22a-4e21-a269-e3eee0954d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:59.210240 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918418f-c22a-4e21-a269-e3eee0954d7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:59.211918 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7918418f-c22a-4e21-a269-e3eee0954d7e None admin] RBAC: Adding query filter params (role.id=23718d83b0864b16b0071e8ffe0b5861, scope.project.id=191069caccd04233b012585e29cfa18b, user.id=98560723b49940d38a6a5e9cf3f5d3b4) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:59.212704 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7918418f-c22a-4e21-a269-e3eee0954d7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:59.215924 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7918418f-c22a-4e21-a269-e3eee0954d7e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:59.231999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 338/662] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:08:58 2022] GET /identity/v3/role_assignments?user.id=98560723b49940d38a6a5e9cf3f5d3b4&scope.project.id=191069caccd04233b012585e29cfa18b&role.id=23718d83b0864b16b0071e8ffe0b5861 => generated 608 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:59.233711 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918418f-c22a-4e21-a269-e3eee0954d7e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:59.277762 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba950fa7-f370-4a81-ac78-53c6efdf68d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:59.279065 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba950fa7-f370-4a81-ac78-53c6efdf68d7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:59.279360 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba950fa7-f370-4a81-ac78-53c6efdf68d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:59.283544 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba950fa7-f370-4a81-ac78-53c6efdf68d7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:59.300684 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba950fa7-f370-4a81-ac78-53c6efdf68d7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:59.302352 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba950fa7-f370-4a81-ac78-53c6efdf68d7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:59.330528 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-ba950fa7-f370-4a81-ac78-53c6efdf68d7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 19:08:59.330528 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:08:59.330528 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:08:59.330528 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:08:59.330528 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:08:59.330528 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:08:59.330528 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:08:59.330528 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:08:59.330528 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:08:59.330528 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:08:59.330528 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:08:59.330528 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:08:59.330528 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:08:59.330528 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:08:59.330528 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:08:59.330528 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:08:59.332705 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:08:59.332705 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:08:59.332705 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:08:59.332705 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:08:59.332705 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:08:59.332705 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:08:59.332705 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:08:59.332705 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:08:59.332705 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:08:59.332705 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:08:59.332705 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:08:59.332705 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:08:59.332705 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:08:59.332705 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:08:59.332705 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:08:59.332705 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:08:59.332705 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:08:59.332705 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:08:59.334987 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:08:59.334987 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:08:59.334987 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:08:59.334987 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:08:59.334987 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:08:59.334987 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:08:59.334987 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:08:59.334987 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:08:59.334987 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 19:08:59.334987 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:08:59.340651 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 325/663] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Jul 25 19:08:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:08:59.340651 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba950fa7-f370-4a81-ac78-53c6efdf68d7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:59.502434 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918418f-c22a-4e21-a269-e3eee0954d7e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:59.550213 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:59.557171 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa395d-fba7-4ba4-8d17-84ed7e7489f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:59.557854 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa395d-fba7-4ba4-8d17-84ed7e7489f0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:59.558137 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa395d-fba7-4ba4-8d17-84ed7e7489f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:59.559126 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa395d-fba7-4ba4-8d17-84ed7e7489f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:59.606326 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba950fa7-f370-4a81-ac78-53c6efdf68d7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:59.661790 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfa395d-fba7-4ba4-8d17-84ed7e7489f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:59.663357 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfa395d-fba7-4ba4-8d17-84ed7e7489f0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:59.684586 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d84fdf-84da-4afa-985c-12124872e6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:08:59.685717 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d84fdf-84da-4afa-985c-12124872e6eb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:08:59.686143 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d84fdf-84da-4afa-985c-12124872e6eb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:08:59.686583 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d84fdf-84da-4afa-985c-12124872e6eb None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:08:59.688096 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d84fdf-84da-4afa-985c-12124872e6eb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:08:59.688658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d84fdf-84da-4afa-985c-12124872e6eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:08:59.691134 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d84fdf-84da-4afa-985c-12124872e6eb None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:08:59.731586 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 326/664] 10.222.0.18 () {56 vars in 1139 bytes} [Mon Jul 25 19:08:59 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:08:59.738704 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d84fdf-84da-4afa-985c-12124872e6eb None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:59.773663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:59.818422 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 339/665] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:08:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 19:08:59.824744 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa395d-fba7-4ba4-8d17-84ed7e7489f0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:08:59.864390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:08:59.960932 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d84fdf-84da-4afa-985c-12124872e6eb None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:08:59.961461 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa395d-fba7-4ba4-8d17-84ed7e7489f0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:00.008188 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ecca3d-d19d-4fd2-bb02-dc2918112af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:00.009262 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ecca3d-d19d-4fd2-bb02-dc2918112af0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:00.009554 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ecca3d-d19d-4fd2-bb02-dc2918112af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:00.009841 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ecca3d-d19d-4fd2-bb02-dc2918112af0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:00.025995 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ecca3d-d19d-4fd2-bb02-dc2918112af0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:00.027779 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ecca3d-d19d-4fd2-bb02-dc2918112af0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:00.033072 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3248684-a274-4c08-ab85-4bf1c4c7ff50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:00.034254 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3248684-a274-4c08-ab85-4bf1c4c7ff50 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:00.034866 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3248684-a274-4c08-ab85-4bf1c4c7ff50 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:00.035505 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3248684-a274-4c08-ab85-4bf1c4c7ff50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:00.049325 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-a3ecca3d-d19d-4fd2-bb02-dc2918112af0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:00.049325 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:00.049325 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:00.049325 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:00.049325 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:00.049325 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:00.049325 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:00.049325 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:00.049325 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:00.049325 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:00.049325 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:00.049325 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:00.049325 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:09:00.049325 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:09:00.049325 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:09:00.050334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:09:00.050334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:00.050334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:00.050334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:00.050334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:00.050334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:00.050334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:00.050334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:00.050334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:00.050334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:00.050334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:00.050334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:00.050334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:00.050334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:00.050334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:00.050334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:00.050334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:00.051267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:09:00.051267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:09:00.051267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:00.051267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:00.051267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 340/666] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:08:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:00.052485 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6909b6-a9aa-4d8a-9cf5-3637f7d53c27 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:00.052784 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6909b6-a9aa-4d8a-9cf5-3637f7d53c27 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:00.053072 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6909b6-a9aa-4d8a-9cf5-3637f7d53c27 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:00.055008 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 341/667] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:00.060610 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6909b6-a9aa-4d8a-9cf5-3637f7d53c27 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:00.187484 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:00.187954 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3248684-a274-4c08-ab85-4bf1c4c7ff50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:00.189786 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3248684-a274-4c08-ab85-4bf1c4c7ff50 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:00.213547 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6909b6-a9aa-4d8a-9cf5-3637f7d53c27 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:00.220494 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:00.267124 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ef6a4f-9a4d-4ab4-a297-3e08974e90b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:00.267903 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef6a4f-9a4d-4ab4-a297-3e08974e90b2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:00.268196 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef6a4f-9a4d-4ab4-a297-3e08974e90b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:00.268477 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef6a4f-9a4d-4ab4-a297-3e08974e90b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:00.269233 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ef6a4f-9a4d-4ab4-a297-3e08974e90b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:00.269550 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ef6a4f-9a4d-4ab4-a297-3e08974e90b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:00.270993 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ef6a4f-9a4d-4ab4-a297-3e08974e90b2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:00.290443 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 342/668] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:09:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:00.292061 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f09f0b-1026-49e4-adf6-b989fd9010c1 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:00.292360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f09f0b-1026-49e4-adf6-b989fd9010c1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:00.293190 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f09f0b-1026-49e4-adf6-b989fd9010c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:00.316371 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-10f09f0b-1026-49e4-adf6-b989fd9010c1 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:00.318613 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-10f09f0b-1026-49e4-adf6-b989fd9010c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:00.354781 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 327/669] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:08:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 19:09:00.354781 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3248684-a274-4c08-ab85-4bf1c4c7ff50 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:00.416923 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 343/670] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:09:00.419249 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f09f0b-1026-49e4-adf6-b989fd9010c1 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:00.514496 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3248684-a274-4c08-ab85-4bf1c4c7ff50 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:00.534827 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:00.538381 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:00.564179 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b48df-ebaf-4c40-a6fd-7b8ad227b612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:00.565278 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b48df-ebaf-4c40-a6fd-7b8ad227b612 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:00.565748 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b48df-ebaf-4c40-a6fd-7b8ad227b612 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:00.566179 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b48df-ebaf-4c40-a6fd-7b8ad227b612 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:00.575263 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f09f0b-1026-49e4-adf6-b989fd9010c1 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:00.575891 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b48df-ebaf-4c40-a6fd-7b8ad227b612 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:00.577820 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b48df-ebaf-4c40-a6fd-7b8ad227b612 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:00.594597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b15b48df-ebaf-4c40-a6fd-7b8ad227b612 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:09:00.594597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:00.594597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:00.594597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:00.594597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:00.594597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:00.594597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:00.594597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:00.594597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:00.594597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:00.594597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:00.594597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:00.594597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:09:00.594597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:09:00.594597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:09:00.595654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:09:00.595654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:00.595654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:00.595654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:00.595654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:00.595654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:00.595654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:00.595654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:00.595654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:00.595654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:00.595654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:00.595654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:00.595654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:00.595654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:00.595654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:00.595654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:00.595654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:00.596613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:09:00.596613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:09:00.596613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:09:00.596613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:09:00.596613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:09:00.596613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:09:00.596613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:09:00.596613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:00.604821 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 328/671] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:09:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:00.606238 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b48df-ebaf-4c40-a6fd-7b8ad227b612 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:00.623987 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5abd91c-ec22-4162-a66d-8bc5ed32dff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:00.625046 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5abd91c-ec22-4162-a66d-8bc5ed32dff5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:00.625466 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5abd91c-ec22-4162-a66d-8bc5ed32dff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:00.625784 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5abd91c-ec22-4162-a66d-8bc5ed32dff5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:00.640559 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5abd91c-ec22-4162-a66d-8bc5ed32dff5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:00.642329 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5abd91c-ec22-4162-a66d-8bc5ed32dff5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:00.665537 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b5abd91c-ec22-4162-a66d-8bc5ed32dff5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:00.665537 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:00.665537 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:00.665537 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:00.665537 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:00.665537 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:00.665537 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:00.665537 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:00.665537 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:00.665537 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:00.665537 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:00.665537 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:00.665537 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:09:00.665537 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:09:00.665537 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:09:00.667035 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:09:00.667035 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:00.667035 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:00.667035 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:00.667035 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:00.667035 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:00.667035 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:00.667035 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:00.667035 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:00.667035 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:00.667035 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:00.667035 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:00.667035 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:00.667035 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:00.667035 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:00.667035 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:00.667035 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:00.668561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:09:00.668561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:09:00.668561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:00.668561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:00.672511 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 344/672] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:09:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:00.673531 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5abd91c-ec22-4162-a66d-8bc5ed32dff5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:00.738071 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b48df-ebaf-4c40-a6fd-7b8ad227b612 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:00.779300 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28b4b7d-748c-4d83-984b-f5d34e750183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:00.780562 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28b4b7d-748c-4d83-984b-f5d34e750183 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:00.781070 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28b4b7d-748c-4d83-984b-f5d34e750183 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:00.781561 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28b4b7d-748c-4d83-984b-f5d34e750183 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:00.795468 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28b4b7d-748c-4d83-984b-f5d34e750183 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:09:00.804287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-f28b4b7d-748c-4d83-984b-f5d34e750183 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:09:00.804287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:00.804287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:00.804287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:00.804287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:00.804287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:00.804287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:00.804287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:00.804287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:00.804287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:00.804287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:00.804287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:00.804287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:09:00.804287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:09:00.804287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:09:00.804287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:09:00.805338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:09:00.805338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:09:00.805338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:09:00.805338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:09:00.805338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:00.805338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:00.805338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:00.805338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:00.805338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:00.805338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:00.805338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:00.805338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:00.805338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:00.805338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:00.805338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:00.805338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:00.806203 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:00.806203 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:00.806203 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:00.806203 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:00.806203 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:09:00.806203 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:09:00.806203 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:09:00.806203 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:09:00.806203 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:09:00.806203 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:09:00.806203 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:09:00.806203 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:00.814264 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5abd91c-ec22-4162-a66d-8bc5ed32dff5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:00.814714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 329/673] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Jul 25 19:09:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:00.815980 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28b4b7d-748c-4d83-984b-f5d34e750183 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:00.849823 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:00.858793 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:00.866446 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e332f8d-3faf-45a9-91a2-b1f9590c6350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:00.867138 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e332f8d-3faf-45a9-91a2-b1f9590c6350 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:00.867489 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e332f8d-3faf-45a9-91a2-b1f9590c6350 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:00.867803 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e332f8d-3faf-45a9-91a2-b1f9590c6350 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:00.868981 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e332f8d-3faf-45a9-91a2-b1f9590c6350 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:00.869147 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e332f8d-3faf-45a9-91a2-b1f9590c6350 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:00.870660 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e332f8d-3faf-45a9-91a2-b1f9590c6350 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:00.893508 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 345/674] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Jul 25 19:09:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:00.893508 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e332f8d-3faf-45a9-91a2-b1f9590c6350 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:00.970687 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28b4b7d-748c-4d83-984b-f5d34e750183 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:01.038998 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e3e23e-8ff3-4c14-9c73-5c2dec1fefa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:01.040594 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3e23e-8ff3-4c14-9c73-5c2dec1fefa7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:01.041308 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3e23e-8ff3-4c14-9c73-5c2dec1fefa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:01.042002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3e23e-8ff3-4c14-9c73-5c2dec1fefa7 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:01.044287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e3e23e-8ff3-4c14-9c73-5c2dec1fefa7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:01.045162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e3e23e-8ff3-4c14-9c73-5c2dec1fefa7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:01.048071 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e3e23e-8ff3-4c14-9c73-5c2dec1fefa7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:01.073494 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 330/675] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:09:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:01.075499 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e3e23e-8ff3-4c14-9c73-5c2dec1fefa7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:01.110598 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e332f8d-3faf-45a9-91a2-b1f9590c6350 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:01.189404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd2f65-1542-4916-b1c1-6b5187aae590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:01.190494 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd2f65-1542-4916-b1c1-6b5187aae590 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:01.190983 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd2f65-1542-4916-b1c1-6b5187aae590 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:01.191460 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd2f65-1542-4916-b1c1-6b5187aae590 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:01.193781 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cd2f65-1542-4916-b1c1-6b5187aae590 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:01.194309 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cd2f65-1542-4916-b1c1-6b5187aae590 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:01.197171 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cd2f65-1542-4916-b1c1-6b5187aae590 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:01.218233 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 346/676] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Jul 25 19:09:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:01.223705 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd2f65-1542-4916-b1c1-6b5187aae590 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:01.287956 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:01.303069 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e3e23e-8ff3-4c14-9c73-5c2dec1fefa7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:01.326130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:01.383914 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ae219-a5ee-4654-9e21-4ead8432a4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:01.385028 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ae219-a5ee-4654-9e21-4ead8432a4b8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:01.386142 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ae219-a5ee-4654-9e21-4ead8432a4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:01.387301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ae219-a5ee-4654-9e21-4ead8432a4b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:01.390014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ae219-a5ee-4654-9e21-4ead8432a4b8 None admin] RBAC: Adding query filter params (role.id=23718d83b0864b16b0071e8ffe0b5861, scope.project.id=191069caccd04233b012585e29cfa18b, user.id=2aa720a35b9844848abcc42a4c4d6a2d) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:01.390559 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ae219-a5ee-4654-9e21-4ead8432a4b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:01.393757 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ae219-a5ee-4654-9e21-4ead8432a4b8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:01.412506 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 331/677] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:09:01 2022] GET /identity/v3/role_assignments?user.id=2aa720a35b9844848abcc42a4c4d6a2d&scope.project.id=191069caccd04233b012585e29cfa18b&role.id=23718d83b0864b16b0071e8ffe0b5861 => generated 608 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:01.412506 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ae219-a5ee-4654-9e21-4ead8432a4b8 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:01.463682 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd2f65-1542-4916-b1c1-6b5187aae590 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:01.558021 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4217eaac-9282-4f02-b744-d989017e17eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:01.560278 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217eaac-9282-4f02-b744-d989017e17eb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:01.560782 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217eaac-9282-4f02-b744-d989017e17eb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:01.561270 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217eaac-9282-4f02-b744-d989017e17eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:01.581398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4217eaac-9282-4f02-b744-d989017e17eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:01.585536 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4217eaac-9282-4f02-b744-d989017e17eb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:01.600312 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ae219-a5ee-4654-9e21-4ead8432a4b8 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:01.625683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-4217eaac-9282-4f02-b744-d989017e17eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:09:01.625683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:01.625683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:01.625683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:01.625683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:01.625683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:01.625683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:01.625683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:01.625683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:01.625683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:01.625683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:01.625683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:01.625683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:09:01.625683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:09:01.625683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:09:01.626808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:09:01.626808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:01.626808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:01.626808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:01.626808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:01.626808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:01.626808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:01.626808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:01.626808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:01.626808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:01.626808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:01.626808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:01.626808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:01.626808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:01.626808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:01.626808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:01.626808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:01.627758 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:09:01.627758 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:09:01.627758 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:09:01.627758 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:09:01.627758 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:09:01.627758 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:09:01.627758 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:09:01.627758 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:01.629194 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 347/678] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:09:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:01.631494 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1410493-c30e-47bc-b674-accb6abd1435 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:01.632023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1410493-c30e-47bc-b674-accb6abd1435 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:01.632482 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1410493-c30e-47bc-b674-accb6abd1435 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:01.637129 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 348/679] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:01.639198 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1410493-c30e-47bc-b674-accb6abd1435 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:01.666756 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05862b8a-17d5-4931-bb8b-22cb50590c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:01.669024 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05862b8a-17d5-4931-bb8b-22cb50590c5e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:01.669434 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05862b8a-17d5-4931-bb8b-22cb50590c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:01.669845 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05862b8a-17d5-4931-bb8b-22cb50590c5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:01.695201 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05862b8a-17d5-4931-bb8b-22cb50590c5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:01.698108 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05862b8a-17d5-4931-bb8b-22cb50590c5e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:01.727680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-05862b8a-17d5-4931-bb8b-22cb50590c5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:01.727680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:01.727680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:01.727680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:01.727680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:01.727680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:01.727680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:01.727680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:01.727680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:01.727680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:01.727680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:01.727680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:01.727680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:09:01.727680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:09:01.727680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:09:01.729775 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:09:01.729775 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:01.729775 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:01.729775 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:01.729775 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:01.729775 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:01.729775 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:01.729775 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:01.729775 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:01.729775 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:01.729775 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:01.729775 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:01.729775 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:01.729775 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:01.729775 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:01.729775 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:01.729775 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:01.731714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:09:01.731714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:09:01.731714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:01.731714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:01.735062 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 332/680] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:09:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:01.737599 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1e5a4-ce34-47ba-817e-5f6ee823015f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:01.738046 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1e5a4-ce34-47ba-817e-5f6ee823015f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:01.738485 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1e5a4-ce34-47ba-817e-5f6ee823015f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:01.773680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-14b1e5a4-ce34-47ba-817e-5f6ee823015f None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:01.776103 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-14b1e5a4-ce34-47ba-817e-5f6ee823015f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:01.783325 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:01.851734 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1410493-c30e-47bc-b674-accb6abd1435 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:01.869592 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:01.894862 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 333/681] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:09:01.895825 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b1e5a4-ce34-47ba-817e-5f6ee823015f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:01.914055 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d97960-3187-40cc-b50c-f23a6ea109cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:01.915198 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d97960-3187-40cc-b50c-f23a6ea109cd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:01.915779 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d97960-3187-40cc-b50c-f23a6ea109cd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:01.916566 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d97960-3187-40cc-b50c-f23a6ea109cd None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:01.918540 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d97960-3187-40cc-b50c-f23a6ea109cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:01.919225 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d97960-3187-40cc-b50c-f23a6ea109cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:01.921153 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d97960-3187-40cc-b50c-f23a6ea109cd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:01.936699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 349/682] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Jul 25 19:09:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:01.938204 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d97960-3187-40cc-b50c-f23a6ea109cd None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:02.025498 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b1e5a4-ce34-47ba-817e-5f6ee823015f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:02.068690 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4aac81-369b-46ba-b553-d082e9da0eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:02.069379 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4aac81-369b-46ba-b553-d082e9da0eea None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:02.069665 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4aac81-369b-46ba-b553-d082e9da0eea None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:02.069945 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4aac81-369b-46ba-b553-d082e9da0eea None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:02.071027 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4aac81-369b-46ba-b553-d082e9da0eea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:02.071345 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4aac81-369b-46ba-b553-d082e9da0eea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:02.072939 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4aac81-369b-46ba-b553-d082e9da0eea None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:02.075789 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d97960-3187-40cc-b50c-f23a6ea109cd None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:02.083944 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:02.089748 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 334/683] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:09:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:02.090901 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4aac81-369b-46ba-b553-d082e9da0eea None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:02.120003 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dfb8f5-e9e0-419a-b8e8-ecd888316ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:02.121493 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfb8f5-e9e0-419a-b8e8-ecd888316ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:02.122109 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfb8f5-e9e0-419a-b8e8-ecd888316ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:02.122606 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfb8f5-e9e0-419a-b8e8-ecd888316ac6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:02.136138 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dfb8f5-e9e0-419a-b8e8-ecd888316ac6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:09:02.145522 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-19dfb8f5-e9e0-419a-b8e8-ecd888316ac6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:09:02.145522 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:02.145522 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:02.145522 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:02.145522 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:02.145522 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:02.145522 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:02.145522 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:02.145522 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:02.145522 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:02.145522 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:02.145522 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:02.145522 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:09:02.145522 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:09:02.145522 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:09:02.145522 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:09:02.146573 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:09:02.146573 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:09:02.146573 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:09:02.146573 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:09:02.146573 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:02.146573 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:02.146573 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:02.146573 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:02.146573 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:02.146573 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:02.146573 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:02.146573 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:02.146573 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:02.146573 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:02.146573 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:02.146573 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:02.147427 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:02.147427 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:02.147427 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:02.147427 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:02.147427 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:09:02.147427 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:09:02.147427 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:09:02.147427 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:09:02.147427 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:09:02.147427 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:09:02.147427 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:09:02.147427 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:02.150466 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 350/684] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Jul 25 19:09:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:02.151635 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dfb8f5-e9e0-419a-b8e8-ecd888316ac6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:02.187911 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:02.232952 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4aac81-369b-46ba-b553-d082e9da0eea None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:02.287150 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a49c98-e207-4e63-869f-4dbf41749c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:02.288188 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a49c98-e207-4e63-869f-4dbf41749c2c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:02.288627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a49c98-e207-4e63-869f-4dbf41749c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:02.289086 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a49c98-e207-4e63-869f-4dbf41749c2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:02.290088 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a49c98-e207-4e63-869f-4dbf41749c2c None admin] RBAC: Adding query filter params (role.id=23718d83b0864b16b0071e8ffe0b5861, scope.project.id=191069caccd04233b012585e29cfa18b, user.id=181c84715c094100b414cea3c4919eb7) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:02.290558 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a49c98-e207-4e63-869f-4dbf41749c2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:02.290817 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dfb8f5-e9e0-419a-b8e8-ecd888316ac6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:02.292615 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a49c98-e207-4e63-869f-4dbf41749c2c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:02.300515 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 335/685] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:09:02 2022] GET /identity/v3/role_assignments?user.id=181c84715c094100b414cea3c4919eb7&scope.project.id=191069caccd04233b012585e29cfa18b&role.id=23718d83b0864b16b0071e8ffe0b5861 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:02.301776 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a49c98-e207-4e63-869f-4dbf41749c2c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:02.349117 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be163b81-3734-4452-8e82-3d0f6fd4d93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:02.349803 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be163b81-3734-4452-8e82-3d0f6fd4d93b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:02.350374 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be163b81-3734-4452-8e82-3d0f6fd4d93b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:02.350937 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be163b81-3734-4452-8e82-3d0f6fd4d93b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:02.366954 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be163b81-3734-4452-8e82-3d0f6fd4d93b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:02.370284 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be163b81-3734-4452-8e82-3d0f6fd4d93b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:02.405904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-be163b81-3734-4452-8e82-3d0f6fd4d93b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 19:09:02.405904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:02.405904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:02.405904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:02.405904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:02.405904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:02.405904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:02.405904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:02.405904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:02.405904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:02.405904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:02.405904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:02.405904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:09:02.405904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:09:02.405904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:09:02.405904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:09:02.407819 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:02.407819 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:02.407819 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:09:02.407819 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:09:02.407819 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:09:02.407819 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:09:02.407819 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:02.407819 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:02.407819 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:02.407819 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:02.407819 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:02.407819 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:02.407819 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:02.407819 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:02.407819 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:02.407819 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:02.407819 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:02.407819 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:02.409691 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:02.409691 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:02.409691 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:09:02.409691 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:09:02.409691 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:09:02.409691 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:09:02.409691 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:09:02.409691 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:09:02.409691 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 19:09:02.409691 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:02.409691 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 351/686] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Jul 25 19:09:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:02.418307 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be163b81-3734-4452-8e82-3d0f6fd4d93b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:02.450400 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:02.535804 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a49c98-e207-4e63-869f-4dbf41749c2c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:02.632734 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0051a74-09b2-4f86-bf27-dd635fd335ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:02.634311 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0051a74-09b2-4f86-bf27-dd635fd335ba None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:02.635059 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0051a74-09b2-4f86-bf27-dd635fd335ba None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:02.635795 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0051a74-09b2-4f86-bf27-dd635fd335ba None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:02.638180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0051a74-09b2-4f86-bf27-dd635fd335ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:02.639071 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0051a74-09b2-4f86-bf27-dd635fd335ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:02.641628 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0051a74-09b2-4f86-bf27-dd635fd335ba None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:02.654938 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:02.664429 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 336/687] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:09:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:02.674068 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0051a74-09b2-4f86-bf27-dd635fd335ba None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:02.682874 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be163b81-3734-4452-8e82-3d0f6fd4d93b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:02.732526 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a127a-f439-465b-a053-c34b0d6a4ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:02.733595 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a127a-f439-465b-a053-c34b0d6a4ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:02.734049 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a127a-f439-465b-a053-c34b0d6a4ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:02.734403 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a127a-f439-465b-a053-c34b0d6a4ac3 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:02.735416 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98a127a-f439-465b-a053-c34b0d6a4ac3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:02.735752 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98a127a-f439-465b-a053-c34b0d6a4ac3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:02.737515 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98a127a-f439-465b-a053-c34b0d6a4ac3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:02.762848 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 352/688] 10.222.0.18 () {56 vars in 1135 bytes} [Mon Jul 25 19:09:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:02.768924 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a127a-f439-465b-a053-c34b0d6a4ac3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:02.901358 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0051a74-09b2-4f86-bf27-dd635fd335ba None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:02.921511 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a127a-f439-465b-a053-c34b0d6a4ac3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:02.936587 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:02.948504 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90f173-1657-4474-9957-56c92d88532a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:02.949814 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90f173-1657-4474-9957-56c92d88532a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:02.950290 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90f173-1657-4474-9957-56c92d88532a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:02.950667 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90f173-1657-4474-9957-56c92d88532a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:02.966774 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90f173-1657-4474-9957-56c92d88532a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:02.968512 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90f173-1657-4474-9957-56c92d88532a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:02.999148 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-6b90f173-1657-4474-9957-56c92d88532a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:02.999148 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:02.999148 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:02.999148 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:02.999148 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:02.999148 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:02.999148 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:02.999148 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:02.999148 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:02.999148 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:02.999148 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:02.999148 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:02.999148 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:09:02.999148 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:09:02.999148 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:09:03.001116 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:09:03.001116 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:03.001116 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:03.001116 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:03.001116 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:03.001116 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:03.001116 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:03.001116 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:03.001116 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:03.001116 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:03.001116 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:03.001116 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:03.001116 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:03.001116 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:03.001116 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:03.001116 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:03.001116 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:03.003011 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:09:03.003011 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:09:03.003011 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:03.003011 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:03.003452 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e1eba-dc17-4eb4-9542-c6191af1ce76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:03.004441 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 337/689] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:09:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:03.004700 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e1eba-dc17-4eb4-9542-c6191af1ce76 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:03.004700 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e1eba-dc17-4eb4-9542-c6191af1ce76 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:03.004700 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e1eba-dc17-4eb4-9542-c6191af1ce76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:03.005279 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b89df4-9e18-484b-a473-4209daba25df None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:03.005279 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b89df4-9e18-484b-a473-4209daba25df None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:03.005279 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b89df4-9e18-484b-a473-4209daba25df None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:03.005279 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 338/690] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:03.015570 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae74a9-f81a-4891-9308-7d87d3c2c82e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:03.015856 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae74a9-f81a-4891-9308-7d87d3c2c82e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:03.016194 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae74a9-f81a-4891-9308-7d87d3c2c82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:03.038987 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-32ae74a9-f81a-4891-9308-7d87d3c2c82e None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:03.040592 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-32ae74a9-f81a-4891-9308-7d87d3c2c82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:03.045871 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:03.135028 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e1eba-dc17-4eb4-9542-c6191af1ce76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:03.137477 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e1eba-dc17-4eb4-9542-c6191af1ce76 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:03.197717 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 339/691] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:09:03.199274 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae74a9-f81a-4891-9308-7d87d3c2c82e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:03.379422 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 353/692] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:09:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 19:09:03.379422 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e1eba-dc17-4eb4-9542-c6191af1ce76 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:03.407281 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae74a9-f81a-4891-9308-7d87d3c2c82e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:03.432391 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:03.463456 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:03.505101 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84e502-b0a1-4234-9bba-cc6dfe862058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:03.506252 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84e502-b0a1-4234-9bba-cc6dfe862058 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:03.506716 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84e502-b0a1-4234-9bba-cc6dfe862058 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:03.507209 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84e502-b0a1-4234-9bba-cc6dfe862058 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:03.509388 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84e502-b0a1-4234-9bba-cc6dfe862058 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:03.509916 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84e502-b0a1-4234-9bba-cc6dfe862058 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:03.514468 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84e502-b0a1-4234-9bba-cc6dfe862058 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:03.540640 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e1eba-dc17-4eb4-9542-c6191af1ce76 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:03.549437 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 340/693] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:09:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:03.550966 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84e502-b0a1-4234-9bba-cc6dfe862058 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:03.595245 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0b919-b209-4c9e-8168-583cf09d9b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:03.596528 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0b919-b209-4c9e-8168-583cf09d9b55 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:03.597028 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0b919-b209-4c9e-8168-583cf09d9b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:03.597586 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0b919-b209-4c9e-8168-583cf09d9b55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:03.617147 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0b919-b209-4c9e-8168-583cf09d9b55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:03.622129 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0b919-b209-4c9e-8168-583cf09d9b55 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:03.649499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-a9c0b919-b209-4c9e-8168-583cf09d9b55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:03.649499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:03.649499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:03.649499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:03.649499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:03.649499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:03.649499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:03.649499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:03.649499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:03.649499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:03.649499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:03.649499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:03.649499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:09:03.649499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:09:03.649499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:09:03.650478 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:09:03.650478 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:03.650478 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:03.650478 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:03.650478 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:03.650478 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:03.650478 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:03.650478 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:03.650478 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:03.650478 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:03.650478 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:03.650478 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:03.650478 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:03.650478 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:03.650478 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:03.650478 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:03.650478 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:03.651381 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:09:03.651381 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:09:03.651381 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:03.651381 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:03.654809 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 354/694] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:09:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:03.656276 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0b919-b209-4c9e-8168-583cf09d9b55 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:03.741171 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84e502-b0a1-4234-9bba-cc6dfe862058 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:03.804108 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8faf41-1dbe-4af2-9067-514b98dead85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:03.806328 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8faf41-1dbe-4af2-9067-514b98dead85 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:03.806835 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8faf41-1dbe-4af2-9067-514b98dead85 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:03.807248 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8faf41-1dbe-4af2-9067-514b98dead85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:03.830946 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8faf41-1dbe-4af2-9067-514b98dead85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:03.833822 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8faf41-1dbe-4af2-9067-514b98dead85 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:03.854260 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:03.861650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-ee8faf41-1dbe-4af2-9067-514b98dead85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:03.861650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:03.861650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:03.861650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:03.861650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:03.861650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:03.861650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:03.861650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:03.861650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:03.861650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:03.861650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:03.861650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:03.861650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:09:03.861650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:09:03.861650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:09:03.862629 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:09:03.862629 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:03.862629 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:03.862629 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:03.862629 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:03.862629 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:03.862629 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:03.862629 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:03.862629 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:03.862629 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:03.862629 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:03.862629 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:03.862629 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:03.862629 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:03.862629 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:03.862629 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:03.862629 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:03.863546 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:09:03.863546 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:09:03.863546 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:03.863546 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:03.863773 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0b919-b209-4c9e-8168-583cf09d9b55 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:03.872889 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 341/695] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:09:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:03.874259 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8faf41-1dbe-4af2-9067-514b98dead85 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:03.905635 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:03.958020 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d2b90-7e3b-429f-825a-0d7527956bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:03.959385 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d2b90-7e3b-429f-825a-0d7527956bbb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:03.959916 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d2b90-7e3b-429f-825a-0d7527956bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:03.960497 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d2b90-7e3b-429f-825a-0d7527956bbb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:03.981862 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d2b90-7e3b-429f-825a-0d7527956bbb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:03.985947 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d2b90-7e3b-429f-825a-0d7527956bbb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:04.027711 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-7c4d2b90-7e3b-429f-825a-0d7527956bbb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 19:09:04.027711 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:04.027711 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:04.027711 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:04.027711 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:04.027711 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:04.027711 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:04.027711 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:04.027711 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:04.027711 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:04.027711 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:04.027711 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:04.027711 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:09:04.027711 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:09:04.027711 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:09:04.027711 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:09:04.029574 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:04.029574 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:04.029574 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:09:04.029574 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:09:04.029574 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:09:04.029574 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:09:04.029574 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:04.029574 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:04.029574 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:04.029574 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:04.029574 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:04.029574 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:04.029574 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:04.029574 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:04.029574 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:04.029574 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:04.029574 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:04.029574 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:04.031480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:04.031480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:04.031480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:09:04.031480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:09:04.031480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:09:04.031480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:09:04.031480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:09:04.031480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:09:04.031480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 19:09:04.031480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:04.036301 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 355/696] 10.222.0.18 () {56 vars in 1093 bytes} [Mon Jul 25 19:09:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:04.038419 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d2b90-7e3b-429f-825a-0d7527956bbb None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:04.051375 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8faf41-1dbe-4af2-9067-514b98dead85 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:04.136664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca498235-3488-4796-bda3-277648a2e9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:04.137800 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca498235-3488-4796-bda3-277648a2e9f9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:04.138291 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca498235-3488-4796-bda3-277648a2e9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:04.138781 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca498235-3488-4796-bda3-277648a2e9f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:04.140523 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca498235-3488-4796-bda3-277648a2e9f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:04.141766 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca498235-3488-4796-bda3-277648a2e9f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:04.145231 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca498235-3488-4796-bda3-277648a2e9f9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:04.181437 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 342/697] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:09:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:04.183038 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca498235-3488-4796-bda3-277648a2e9f9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:04.255953 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d2b90-7e3b-429f-825a-0d7527956bbb None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:04.343913 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db16a72f-f50f-425b-9554-0339c1686b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:04.344606 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16a72f-f50f-425b-9554-0339c1686b90 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:04.345094 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16a72f-f50f-425b-9554-0339c1686b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:04.345518 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16a72f-f50f-425b-9554-0339c1686b90 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:04.346274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db16a72f-f50f-425b-9554-0339c1686b90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:04.346589 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db16a72f-f50f-425b-9554-0339c1686b90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:04.348291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db16a72f-f50f-425b-9554-0339c1686b90 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:04.373550 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 356/698] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:09:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:04.373550 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db16a72f-f50f-425b-9554-0339c1686b90 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:04.375550 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:04.401378 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:04.408693 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca498235-3488-4796-bda3-277648a2e9f9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:04.466920 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcba9cbb-6373-405c-b59d-5f44eb2d60b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:04.467823 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba9cbb-6373-405c-b59d-5f44eb2d60b8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:04.468331 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba9cbb-6373-405c-b59d-5f44eb2d60b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:04.468844 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba9cbb-6373-405c-b59d-5f44eb2d60b8 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:04.470508 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcba9cbb-6373-405c-b59d-5f44eb2d60b8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:04.471071 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcba9cbb-6373-405c-b59d-5f44eb2d60b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:04.473976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcba9cbb-6373-405c-b59d-5f44eb2d60b8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:04.512106 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 343/699] 10.222.0.18 () {56 vars in 1133 bytes} [Mon Jul 25 19:09:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:04.514128 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc502c-ef5d-4415-bee7-c46e8208e99c None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:04.514653 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc502c-ef5d-4415-bee7-c46e8208e99c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:04.515176 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc502c-ef5d-4415-bee7-c46e8208e99c None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:04.521797 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 344/700] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:04.521797 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc502c-ef5d-4415-bee7-c46e8208e99c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:04.560259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db16a72f-f50f-425b-9554-0339c1686b90 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:04.621945 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e97edd-8bfc-448b-a1ea-699309488e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:04.622983 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e97edd-8bfc-448b-a1ea-699309488e1a None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:04.623446 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e97edd-8bfc-448b-a1ea-699309488e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:04.623982 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e97edd-8bfc-448b-a1ea-699309488e1a None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:04.625295 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e97edd-8bfc-448b-a1ea-699309488e1a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:04.627992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e97edd-8bfc-448b-a1ea-699309488e1a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:04.661662 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 357/701] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Jul 25 19:09:04 2022] POST /identity/v3/users => generated 254 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:09:04.662539 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e97edd-8bfc-448b-a1ea-699309488e1a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:04.733932 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc502c-ef5d-4415-bee7-c46e8208e99c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:04.799850 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37df3b2f-36fa-4f9c-afaf-356e5dddaac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:04.801276 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df3b2f-36fa-4f9c-afaf-356e5dddaac5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:04.801900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df3b2f-36fa-4f9c-afaf-356e5dddaac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:04.802879 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df3b2f-36fa-4f9c-afaf-356e5dddaac5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:04.815557 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37df3b2f-36fa-4f9c-afaf-356e5dddaac5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:04.818173 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37df3b2f-36fa-4f9c-afaf-356e5dddaac5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:04.821792 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:04.844270 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-37df3b2f-36fa-4f9c-afaf-356e5dddaac5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:09:04.844270 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:04.844270 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:04.844270 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:04.844270 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:04.844270 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:04.844270 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:04.844270 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:04.844270 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:04.844270 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:04.844270 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:04.844270 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:04.844270 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:09:04.844270 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:09:04.844270 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:09:04.845285 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:09:04.845285 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:04.845285 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:04.845285 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:04.845285 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:04.845285 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:04.845285 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:04.845285 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:04.845285 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:04.845285 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:04.845285 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:04.845285 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:04.845285 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:04.845285 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:04.845285 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:04.845285 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:04.845285 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:04.846327 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:09:04.846327 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:09:04.846327 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:09:04.846327 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:09:04.846327 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:09:04.846327 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:09:04.846327 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:09:04.846327 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:04.849990 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:04.856491 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 345/702] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:09:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:04.860599 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64222fb-df8f-4cd9-acda-4215ad2da93e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:04.862312 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64222fb-df8f-4cd9-acda-4215ad2da93e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:04.862312 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64222fb-df8f-4cd9-acda-4215ad2da93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:04.888385 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a64222fb-df8f-4cd9-acda-4215ad2da93e None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:04.890585 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a64222fb-df8f-4cd9-acda-4215ad2da93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:04.908830 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e97edd-8bfc-448b-a1ea-699309488e1a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:04.970940 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34495082-86bc-4a60-8e44-c61fb86b9467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:04.971831 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34495082-86bc-4a60-8e44-c61fb86b9467 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:04.972284 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34495082-86bc-4a60-8e44-c61fb86b9467 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:04.972758 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34495082-86bc-4a60-8e44-c61fb86b9467 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:05.000710 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 346/703] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:09:05.002081 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64222fb-df8f-4cd9-acda-4215ad2da93e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:05.115218 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34495082-86bc-4a60-8e44-c61fb86b9467 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:05.118206 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34495082-86bc-4a60-8e44-c61fb86b9467 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:05.190092 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64222fb-df8f-4cd9-acda-4215ad2da93e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:05.229673 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:05.238892 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:05.245850 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df398a-0ed4-493e-9b1f-0b6f4368fb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:05.247014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df398a-0ed4-493e-9b1f-0b6f4368fb8c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:05.247437 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df398a-0ed4-493e-9b1f-0b6f4368fb8c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:05.247865 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df398a-0ed4-493e-9b1f-0b6f4368fb8c None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:05.249425 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df398a-0ed4-493e-9b1f-0b6f4368fb8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:05.249992 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df398a-0ed4-493e-9b1f-0b6f4368fb8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:05.252773 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df398a-0ed4-493e-9b1f-0b6f4368fb8c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:05.272163 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 347/704] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Jul 25 19:09:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:05.273348 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df398a-0ed4-493e-9b1f-0b6f4368fb8c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:05.292122 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 358/705] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:09:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 19:09:05.292122 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c207e-2897-40fa-a66b-bc44b3fb4567 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:05.292122 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c207e-2897-40fa-a66b-bc44b3fb4567 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:05.292122 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c207e-2897-40fa-a66b-bc44b3fb4567 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:05.296349 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 359/706] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:05.303482 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271c207e-2897-40fa-a66b-bc44b3fb4567 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:05.448745 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df398a-0ed4-493e-9b1f-0b6f4368fb8c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:05.492742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271c207e-2897-40fa-a66b-bc44b3fb4567 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:05.521307 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7defa766-d13d-484b-b123-58d37148b9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:05.523114 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7defa766-d13d-484b-b123-58d37148b9a6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:05.524407 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7defa766-d13d-484b-b123-58d37148b9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:05.524407 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7defa766-d13d-484b-b123-58d37148b9a6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:05.525487 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7defa766-d13d-484b-b123-58d37148b9a6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:05.528327 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7defa766-d13d-484b-b123-58d37148b9a6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:05.550590 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-7defa766-d13d-484b-b123-58d37148b9a6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 19:09:05.550590 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:05.550590 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 19:09:05.550590 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 19:09:05.550590 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 19:09:05.550590 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 19:09:05.550590 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 19:09:05.550590 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:05.550590 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 19:09:05.550590 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:05.550590 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 19:09:05.550590 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:05.550590 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:05.550590 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:05.550590 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:05.550590 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:05.552572 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:05.552572 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:05.552572 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:05.552572 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:05.552572 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:05.552572 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:05.552572 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:05.552572 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 19:09:05.552572 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 19:09:05.552572 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 19:09:05.552572 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 19:09:05.552572 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:05.552572 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:05.552572 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:05.552572 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:05.552572 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:05.554056 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:05.554056 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:05.554056 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:05.554056 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:05.554056 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:05.554056 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:05.554056 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:05.554056 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:05.554056 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:05.554056 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:05.554056 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:05.554056 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 19:09:05.554056 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:05.554056 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 19:09:05.554056 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:05.554056 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 348/707] 10.222.0.18 () {56 vars in 1129 bytes} [Mon Jul 25 19:09:05 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:05.557413 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7defa766-d13d-484b-b123-58d37148b9a6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:05.557643 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8347620-86ab-4b7b-b4fc-a330323a5599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:05.558003 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8347620-86ab-4b7b-b4fc-a330323a5599 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:05.558859 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8347620-86ab-4b7b-b4fc-a330323a5599 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:05.559990 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8347620-86ab-4b7b-b4fc-a330323a5599 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:05.560822 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8347620-86ab-4b7b-b4fc-a330323a5599 None admin] RBAC: Adding query filter params (role.id=23718d83b0864b16b0071e8ffe0b5861, scope.project.id=191069caccd04233b012585e29cfa18b, user.id=feafcf1b5a9844fe9738823fddd07e0e) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:05.561131 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8347620-86ab-4b7b-b4fc-a330323a5599 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:05.562882 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8347620-86ab-4b7b-b4fc-a330323a5599 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:05.578951 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 360/708] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:09:05 2022] GET /identity/v3/role_assignments?user.id=feafcf1b5a9844fe9738823fddd07e0e&scope.project.id=191069caccd04233b012585e29cfa18b&role.id=23718d83b0864b16b0071e8ffe0b5861 => generated 608 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:05.578951 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d7df3-0b3d-4924-a926-7a9760c0c1be None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:05.578951 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d7df3-0b3d-4924-a926-7a9760c0c1be None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:05.578951 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d7df3-0b3d-4924-a926-7a9760c0c1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:05.599186 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6b5d7df3-0b3d-4924-a926-7a9760c0c1be None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:05.600751 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6b5d7df3-0b3d-4924-a926-7a9760c0c1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:05.658563 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:05.687788 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:05.728167 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 361/709] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:09:05.729496 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d7df3-0b3d-4924-a926-7a9760c0c1be None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:05.817440 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7defa766-d13d-484b-b123-58d37148b9a6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:05.875762 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e6d71-03d2-4017-805d-a558542de537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:05.876590 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e6d71-03d2-4017-805d-a558542de537 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:05.876880 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e6d71-03d2-4017-805d-a558542de537 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:05.877160 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e6d71-03d2-4017-805d-a558542de537 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:05.893521 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e6d71-03d2-4017-805d-a558542de537 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:05.895368 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e6d71-03d2-4017-805d-a558542de537 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:05.909819 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d7df3-0b3d-4924-a926-7a9760c0c1be None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:05.916805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-200e6d71-03d2-4017-805d-a558542de537 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:05.916805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:05.916805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:05.916805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:05.916805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:05.916805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:05.916805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:05.916805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:05.916805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:05.916805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:05.916805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:05.916805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:05.916805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:09:05.916805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:09:05.916805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:09:05.917886 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:09:05.917886 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:05.917886 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:05.917886 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:05.917886 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:05.917886 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:05.917886 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:05.917886 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:05.917886 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:05.917886 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:05.917886 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:05.917886 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:05.917886 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:05.917886 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:05.917886 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:05.917886 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:05.917886 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:05.918825 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:09:05.918825 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:09:05.918825 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:05.918825 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:05.920213 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 349/710] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:09:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:05.921761 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e6d71-03d2-4017-805d-a558542de537 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:05.967286 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2304b9-ff4b-49f7-b80e-8cfc5a9afcd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:05.968209 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2304b9-ff4b-49f7-b80e-8cfc5a9afcd9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:05.968643 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2304b9-ff4b-49f7-b80e-8cfc5a9afcd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:05.969011 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2304b9-ff4b-49f7-b80e-8cfc5a9afcd9 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:05.969971 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2304b9-ff4b-49f7-b80e-8cfc5a9afcd9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:05.970331 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2304b9-ff4b-49f7-b80e-8cfc5a9afcd9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:05.972620 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2304b9-ff4b-49f7-b80e-8cfc5a9afcd9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:05.981882 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 362/711] 10.222.0.18 () {56 vars in 1124 bytes} [Mon Jul 25 19:09:05 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:05.982985 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2304b9-ff4b-49f7-b80e-8cfc5a9afcd9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:06.029299 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:06.104042 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e6d71-03d2-4017-805d-a558542de537 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:06.109144 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:06.145021 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2304b9-ff4b-49f7-b80e-8cfc5a9afcd9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:06.151369 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a382e9da-8817-437e-8770-dbe2a23e6a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:06.153891 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382e9da-8817-437e-8770-dbe2a23e6a04 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:06.154533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382e9da-8817-437e-8770-dbe2a23e6a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:06.155034 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382e9da-8817-437e-8770-dbe2a23e6a04 None admin] PATH_INFO: `/v3/services/network` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:06.155997 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a382e9da-8817-437e-8770-dbe2a23e6a04 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:06.157846 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a382e9da-8817-437e-8770-dbe2a23e6a04 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:06.172216 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-a382e9da-8817-437e-8770-dbe2a23e6a04 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 19:09:06.172216 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:06.172216 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 19:09:06.172216 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 19:09:06.172216 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 19:09:06.172216 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 19:09:06.172216 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 19:09:06.172216 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:06.172216 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 19:09:06.172216 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:06.172216 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 19:09:06.172216 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:06.172216 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:06.172216 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:06.172216 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:06.172216 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:06.172216 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:06.173504 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:06.173504 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:06.173504 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:06.173504 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:06.173504 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:06.173504 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:06.173504 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 19:09:06.173504 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 19:09:06.173504 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 19:09:06.173504 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 19:09:06.173504 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:06.173504 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:06.173504 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:06.173504 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:06.173504 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:06.173504 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:06.174620 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:06.174620 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:06.174620 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:06.174620 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:06.174620 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:06.174620 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:06.174620 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:06.174620 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:06.174620 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:06.174620 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:06.174620 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 19:09:06.174620 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:06.174620 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 19:09:06.174620 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:06.180768 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 350/712] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:09:05 2022] GET /identity/v3/services/network => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:06.181895 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a382e9da-8817-437e-8770-dbe2a23e6a04 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:06.210272 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecbef55-7b18-45bc-bbda-d3368d23ae3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:06.211239 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecbef55-7b18-45bc-bbda-d3368d23ae3b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:06.211563 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecbef55-7b18-45bc-bbda-d3368d23ae3b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:06.211850 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecbef55-7b18-45bc-bbda-d3368d23ae3b None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:06.212619 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecbef55-7b18-45bc-bbda-d3368d23ae3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:06.212914 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecbef55-7b18-45bc-bbda-d3368d23ae3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:06.215844 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecbef55-7b18-45bc-bbda-d3368d23ae3b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:06.239637 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 363/713] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:09:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:06.239637 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecbef55-7b18-45bc-bbda-d3368d23ae3b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:06.363555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a382e9da-8817-437e-8770-dbe2a23e6a04 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:06.408116 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:06.415988 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecbef55-7b18-45bc-bbda-d3368d23ae3b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:06.442918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6739c-b7a1-4a62-af92-b7745483521d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:06.443983 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6739c-b7a1-4a62-af92-b7745483521d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:06.444494 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6739c-b7a1-4a62-af92-b7745483521d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:06.445005 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6739c-b7a1-4a62-af92-b7745483521d None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:06.446523 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa6739c-b7a1-4a62-af92-b7745483521d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:06.447103 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa6739c-b7a1-4a62-af92-b7745483521d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:06.451442 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa6739c-b7a1-4a62-af92-b7745483521d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:06.467566 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 351/714] 10.222.0.18 () {56 vars in 1124 bytes} [Mon Jul 25 19:09:06 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:06.467566 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6739c-b7a1-4a62-af92-b7745483521d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:06.507778 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07ce75-ebc3-4c52-afab-cc256235b76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:06.510072 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07ce75-ebc3-4c52-afab-cc256235b76a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:06.510289 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07ce75-ebc3-4c52-afab-cc256235b76a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:06.510617 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07ce75-ebc3-4c52-afab-cc256235b76a None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:06.511292 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07ce75-ebc3-4c52-afab-cc256235b76a None admin] RBAC: Adding query filter params (name=network) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:06.512329 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07ce75-ebc3-4c52-afab-cc256235b76a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:06.516164 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07ce75-ebc3-4c52-afab-cc256235b76a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:06.531771 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 364/715] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Jul 25 19:09:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:06.538557 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07ce75-ebc3-4c52-afab-cc256235b76a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:06.552189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:06.739109 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6739c-b7a1-4a62-af92-b7745483521d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:06.785038 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07ce75-ebc3-4c52-afab-cc256235b76a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:06.820960 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4355dc-2549-499b-b4e3-adbfc0d48eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:06.822996 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4355dc-2549-499b-b4e3-adbfc0d48eb6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:06.824737 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4355dc-2549-499b-b4e3-adbfc0d48eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:06.825517 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4355dc-2549-499b-b4e3-adbfc0d48eb6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:06.850178 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4355dc-2549-499b-b4e3-adbfc0d48eb6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:06.853621 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4355dc-2549-499b-b4e3-adbfc0d48eb6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:06.892944 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575a134-c1dc-4036-b001-553de2317b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:06.894532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575a134-c1dc-4036-b001-553de2317b0f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:06.900709 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575a134-c1dc-4036-b001-553de2317b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:06.901589 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575a134-c1dc-4036-b001-553de2317b0f None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:06.904167 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0575a134-c1dc-4036-b001-553de2317b0f None admin] RBAC: Adding query filter params (type=network) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:06.905044 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0575a134-c1dc-4036-b001-553de2317b0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:06.911107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-de4355dc-2549-499b-b4e3-adbfc0d48eb6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:09:06.911107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:06.911107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:06.911107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:06.911107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:06.911107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:06.911107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:06.911107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:06.911107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:06.911107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:06.911107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:06.911107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:06.911107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:09:06.911107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:09:06.911107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:09:06.913052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:09:06.913052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:06.913052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:06.913052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:06.913052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:06.913052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:06.913052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:06.913052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:06.913052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:06.913052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:06.913052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:06.913052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:06.913052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:06.913052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:06.913052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:06.913052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:06.913052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:06.914747 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:09:06.914747 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:09:06.914747 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:09:06.914747 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:09:06.914747 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:09:06.914747 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:09:06.914747 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:09:06.914747 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:06.916503 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0575a134-c1dc-4036-b001-553de2317b0f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:06.922036 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 352/716] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:09:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 461 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:06.933920 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4355dc-2549-499b-b4e3-adbfc0d48eb6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:06.947027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 365/717] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Jul 25 19:09:06 2022] GET /identity/v3/services?type=network => generated 124 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:09:07.184543 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:07.218772 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4355dc-2549-499b-b4e3-adbfc0d48eb6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:07.290022 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b41b05-52a0-4500-8659-cf4c27663033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:07.291148 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b41b05-52a0-4500-8659-cf4c27663033 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:07.291651 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b41b05-52a0-4500-8659-cf4c27663033 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:07.292111 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b41b05-52a0-4500-8659-cf4c27663033 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:07.294042 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b41b05-52a0-4500-8659-cf4c27663033 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:07.294550 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b41b05-52a0-4500-8659-cf4c27663033 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:07.297185 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b41b05-52a0-4500-8659-cf4c27663033 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:07.322811 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 353/718] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Jul 25 19:09:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:09:07.329789 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575a134-c1dc-4036-b001-553de2317b0f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:07.381668 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:07.540561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575a134-c1dc-4036-b001-553de2317b0f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:07.616682 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2e7ad0-fd7f-4c71-baaf-f0cd89cb5a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:07.617837 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e7ad0-fd7f-4c71-baaf-f0cd89cb5a0a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:07.618347 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e7ad0-fd7f-4c71-baaf-f0cd89cb5a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:07.618843 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e7ad0-fd7f-4c71-baaf-f0cd89cb5a0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:07.622467 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2e7ad0-fd7f-4c71-baaf-f0cd89cb5a0a None admin] RBAC: Adding query filter params (role.id=ed7fec18968241efa76a0ee769d96da9, scope.project.id=191069caccd04233b012585e29cfa18b, user.id=860e7d26ad364795bb25836aadd4ac7b) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:07.623052 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2e7ad0-fd7f-4c71-baaf-f0cd89cb5a0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:07.625947 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2e7ad0-fd7f-4c71-baaf-f0cd89cb5a0a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:07.642686 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 366/719] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:09:07 2022] GET /identity/v3/role_assignments?user.id=860e7d26ad364795bb25836aadd4ac7b&scope.project.id=191069caccd04233b012585e29cfa18b&role.id=ed7fec18968241efa76a0ee769d96da9 => generated 259 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:08.333633 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978c277-9422-4aa8-8b5a-62cb712b1685 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:08.335218 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978c277-9422-4aa8-8b5a-62cb712b1685 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:08.336048 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978c277-9422-4aa8-8b5a-62cb712b1685 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:08.339564 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 354/720] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:08.346805 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd841c8e-4854-420c-9362-030ab3dc4caa None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:08.348464 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd841c8e-4854-420c-9362-030ab3dc4caa None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:08.349289 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd841c8e-4854-420c-9362-030ab3dc4caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:08.377793 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-fd841c8e-4854-420c-9362-030ab3dc4caa None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:08.379173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fd841c8e-4854-420c-9362-030ab3dc4caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:08.543968 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 367/721] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:09:08.624763 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978c277-9422-4aa8-8b5a-62cb712b1685 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:08.804335 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978c277-9422-4aa8-8b5a-62cb712b1685 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:08.865541 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa9c1c-efcf-4545-ba78-5a8f7e6e0326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:08.866384 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa9c1c-efcf-4545-ba78-5a8f7e6e0326 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:08.866666 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa9c1c-efcf-4545-ba78-5a8f7e6e0326 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:08.866955 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa9c1c-efcf-4545-ba78-5a8f7e6e0326 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:08.881403 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fa9c1c-efcf-4545-ba78-5a8f7e6e0326 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:09:08.891015 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-a9fa9c1c-efcf-4545-ba78-5a8f7e6e0326 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:09:08.891015 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:08.891015 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:08.891015 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:08.891015 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:08.891015 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:08.891015 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:08.891015 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:08.891015 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:08.891015 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:08.891015 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:08.891015 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:08.891015 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:09:08.891015 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:09:08.891015 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:09:08.891015 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:09:08.892925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:09:08.892925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:09:08.892925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:09:08.892925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:09:08.892925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:08.892925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:08.892925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:08.892925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:08.892925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:08.892925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:08.892925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:08.892925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:08.892925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:08.892925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:08.892925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:08.892925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:08.894627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:08.894627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:08.894627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:08.894627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:08.894627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:09:08.894627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:09:08.894627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:09:08.894627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:09:08.894627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:09:08.894627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:09:08.894627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:09:08.894627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:08.894627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 355/722] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Jul 25 19:09:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:08.907353 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd841c8e-4854-420c-9362-030ab3dc4caa None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:08.952140 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:09.180190 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd841c8e-4854-420c-9362-030ab3dc4caa None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:09.248088 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb13a6-ff3e-4f82-82be-e6f33fa8a3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:09.248088 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb13a6-ff3e-4f82-82be-e6f33fa8a3b4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:09.248088 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb13a6-ff3e-4f82-82be-e6f33fa8a3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:09.248088 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb13a6-ff3e-4f82-82be-e6f33fa8a3b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:09.248777 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eb13a6-ff3e-4f82-82be-e6f33fa8a3b4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:09.248777 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eb13a6-ff3e-4f82-82be-e6f33fa8a3b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:09.250120 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eb13a6-ff3e-4f82-82be-e6f33fa8a3b4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:09.275797 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 368/723] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Jul 25 19:09:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:09.284889 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa9c1c-efcf-4545-ba78-5a8f7e6e0326 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:09.306971 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:09.382177 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae3682-b91f-4fac-96b3-f923ab06b6bf None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:09.383009 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae3682-b91f-4fac-96b3-f923ab06b6bf None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:09.383009 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae3682-b91f-4fac-96b3-f923ab06b6bf None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:09.384645 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 369/724] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:09.393618 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6221a-9227-497e-a741-76ba17c89421 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:09.394152 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6221a-9227-497e-a741-76ba17c89421 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:09.394416 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6221a-9227-497e-a741-76ba17c89421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:09.433765 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-efb6221a-9227-497e-a741-76ba17c89421 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:09.435027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-efb6221a-9227-497e-a741-76ba17c89421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:09.522173 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa9c1c-efcf-4545-ba78-5a8f7e6e0326 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:09.621912 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414fe3b7-e015-4888-bbd4-e71359105c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:09.623329 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414fe3b7-e015-4888-bbd4-e71359105c8f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:09.623878 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414fe3b7-e015-4888-bbd4-e71359105c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:09.624550 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414fe3b7-e015-4888-bbd4-e71359105c8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:09.646435 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414fe3b7-e015-4888-bbd4-e71359105c8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:09.649120 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414fe3b7-e015-4888-bbd4-e71359105c8f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:09.682063 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 370/725] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:09:09.688831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-414fe3b7-e015-4888-bbd4-e71359105c8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:09.688831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:09.688831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:09.688831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:09.688831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:09.688831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:09.688831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:09.688831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:09.688831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:09.688831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:09.688831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:09.688831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:09.688831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:09:09.688831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:09:09.688831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:09:09.690608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:09:09.690608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:09.690608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:09.690608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:09.690608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:09.690608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:09.690608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:09.690608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:09.690608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:09.690608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:09.690608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:09.690608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:09.690608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:09.690608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:09.690608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:09.690608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:09.690608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:09.692398 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:09:09.692398 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:09:09.692398 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:09.692398 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:09.692398 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 356/726] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:09:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 407 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:09.703979 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6221a-9227-497e-a741-76ba17c89421 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:09.725303 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:09.749119 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414fe3b7-e015-4888-bbd4-e71359105c8f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:09.941457 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6221a-9227-497e-a741-76ba17c89421 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:09.969650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:10.023190 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dab812-369c-4481-8829-ca3e79ece18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:10.028596 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dab812-369c-4481-8829-ca3e79ece18a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:10.029108 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dab812-369c-4481-8829-ca3e79ece18a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:10.029618 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dab812-369c-4481-8829-ca3e79ece18a None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:10.031044 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dab812-369c-4481-8829-ca3e79ece18a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:10.031635 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dab812-369c-4481-8829-ca3e79ece18a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:10.039714 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dab812-369c-4481-8829-ca3e79ece18a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:10.085554 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 371/727] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:09:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:10.096574 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dab812-369c-4481-8829-ca3e79ece18a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:10.111309 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414fe3b7-e015-4888-bbd4-e71359105c8f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:10.222540 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95638883-e96b-4a11-82d1-5750a1e2a05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:10.223861 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95638883-e96b-4a11-82d1-5750a1e2a05d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:10.224373 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95638883-e96b-4a11-82d1-5750a1e2a05d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:10.224862 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95638883-e96b-4a11-82d1-5750a1e2a05d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:10.260507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95638883-e96b-4a11-82d1-5750a1e2a05d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:09:10.279561 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-95638883-e96b-4a11-82d1-5750a1e2a05d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:09:10.279561 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:10.279561 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:10.279561 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:10.279561 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:10.279561 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:10.279561 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:10.279561 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:10.279561 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:10.279561 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:10.279561 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:10.279561 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:10.279561 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:09:10.279561 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:09:10.279561 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:09:10.279561 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:09:10.281486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:09:10.281486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:09:10.281486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:09:10.281486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:09:10.281486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:10.281486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:10.281486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:10.281486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:10.281486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:10.281486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:10.281486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:10.281486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:10.281486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:10.281486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:10.281486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:10.281486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:10.283084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:10.283084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:10.283084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:10.283084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:10.283084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:09:10.283084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:09:10.283084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:09:10.283084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:09:10.283084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:09:10.283084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:09:10.283084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:09:10.283084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:10.283084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 357/728] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Jul 25 19:09:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 532 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:10.297120 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95638883-e96b-4a11-82d1-5750a1e2a05d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:10.325841 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:10.420199 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dab812-369c-4481-8829-ca3e79ece18a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:10.528173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19e2b7b-36f1-4193-9ffe-6b6475a1883f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:10.530122 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e2b7b-36f1-4193-9ffe-6b6475a1883f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:10.531023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e2b7b-36f1-4193-9ffe-6b6475a1883f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:10.532000 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e2b7b-36f1-4193-9ffe-6b6475a1883f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:10.550561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19e2b7b-36f1-4193-9ffe-6b6475a1883f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:10.554164 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19e2b7b-36f1-4193-9ffe-6b6475a1883f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:10.582309 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95638883-e96b-4a11-82d1-5750a1e2a05d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:10.588955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e19e2b7b-36f1-4193-9ffe-6b6475a1883f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 19:09:10.588955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:10.588955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:10.588955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:10.588955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:10.588955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:10.588955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:10.588955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:10.588955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:10.588955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:10.588955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:10.588955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:10.588955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:09:10.588955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:09:10.588955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:09:10.588955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:09:10.590923 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:10.590923 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:10.590923 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:09:10.590923 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:09:10.590923 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:09:10.590923 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:09:10.590923 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:10.590923 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:10.590923 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:10.590923 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:10.590923 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:10.590923 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:10.590923 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:10.590923 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:10.590923 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:10.590923 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:10.590923 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:10.590923 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:10.593149 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:10.593149 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:10.593149 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:09:10.593149 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:09:10.593149 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:09:10.593149 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:09:10.593149 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:09:10.593149 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:09:10.593149 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 19:09:10.593149 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:10.604035 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 372/729] 10.222.0.18 () {56 vars in 1117 bytes} [Mon Jul 25 19:09:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 509 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:10.607243 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411010ae-54df-4121-b6c7-1991e4948a89 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:10.608233 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411010ae-54df-4121-b6c7-1991e4948a89 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:10.609030 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411010ae-54df-4121-b6c7-1991e4948a89 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:10.616686 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 373/730] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:10 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:10.618719 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411010ae-54df-4121-b6c7-1991e4948a89 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:10.628782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d6608b-003c-439e-bc7d-6953a153aa4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:10.628782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6608b-003c-439e-bc7d-6953a153aa4e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:10.628782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6608b-003c-439e-bc7d-6953a153aa4e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:10.628782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6608b-003c-439e-bc7d-6953a153aa4e None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:10.629900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d6608b-003c-439e-bc7d-6953a153aa4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:10.630226 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d6608b-003c-439e-bc7d-6953a153aa4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:10.631781 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d6608b-003c-439e-bc7d-6953a153aa4e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:10.639754 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:10.644556 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 358/731] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:09:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:09:10.646135 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b5284c-8ef4-4ed3-a806-f84328948197 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:10.646433 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b5284c-8ef4-4ed3-a806-f84328948197 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:10.646705 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b5284c-8ef4-4ed3-a806-f84328948197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:10.671314 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f6b5284c-8ef4-4ed3-a806-f84328948197 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:10.672749 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f6b5284c-8ef4-4ed3-a806-f84328948197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:10.787714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 359/732] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:09:10.787714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b5284c-8ef4-4ed3-a806-f84328948197 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:10.901433 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411010ae-54df-4121-b6c7-1991e4948a89 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:10.958692 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:10.969009 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b5284c-8ef4-4ed3-a806-f84328948197 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:10.975653 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb96382-77b6-47aa-a3d7-83a6ae5a9459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:10.976622 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb96382-77b6-47aa-a3d7-83a6ae5a9459 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:10.976940 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb96382-77b6-47aa-a3d7-83a6ae5a9459 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:10.977223 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb96382-77b6-47aa-a3d7-83a6ae5a9459 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:10.978687 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb96382-77b6-47aa-a3d7-83a6ae5a9459 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:10.979219 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb96382-77b6-47aa-a3d7-83a6ae5a9459 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:10.982713 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb96382-77b6-47aa-a3d7-83a6ae5a9459 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:11.011074 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:11.023557 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 374/733] 10.222.0.18 () {56 vars in 1149 bytes} [Mon Jul 25 19:09:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:11.023557 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730012f3-ada8-4c19-bfc9-943dc8084e7c None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:11.023557 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730012f3-ada8-4c19-bfc9-943dc8084e7c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:11.023557 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730012f3-ada8-4c19-bfc9-943dc8084e7c None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:11.025598 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 375/734] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:11.026178 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730012f3-ada8-4c19-bfc9-943dc8084e7c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:11.059768 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950bc592-f323-44da-b7a7-eb3f0c3cd8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:11.060606 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950bc592-f323-44da-b7a7-eb3f0c3cd8e8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:11.060901 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950bc592-f323-44da-b7a7-eb3f0c3cd8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:11.061185 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950bc592-f323-44da-b7a7-eb3f0c3cd8e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:11.075783 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950bc592-f323-44da-b7a7-eb3f0c3cd8e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:11.077506 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950bc592-f323-44da-b7a7-eb3f0c3cd8e8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:11.124566 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-950bc592-f323-44da-b7a7-eb3f0c3cd8e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:11.124566 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:11.124566 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:11.124566 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:11.124566 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:11.124566 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:11.124566 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:11.124566 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:11.124566 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:11.124566 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:11.124566 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:11.124566 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:11.124566 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:09:11.124566 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:09:11.124566 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:09:11.126080 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:09:11.126080 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:11.126080 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:11.126080 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:11.126080 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:11.126080 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:11.126080 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:11.126080 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:11.126080 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:11.126080 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:11.126080 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:11.126080 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:11.126080 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:11.126080 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:11.126080 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:11.126080 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:11.126080 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:11.128389 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:09:11.128389 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:09:11.128389 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:11.128389 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:11.128389 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 360/735] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:09:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:11.129174 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950bc592-f323-44da-b7a7-eb3f0c3cd8e8 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:11.253184 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730012f3-ada8-4c19-bfc9-943dc8084e7c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:11.310832 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e453a40-a2dd-4aa0-ad72-4798ac40670b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:11.311654 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e453a40-a2dd-4aa0-ad72-4798ac40670b None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:11.312040 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e453a40-a2dd-4aa0-ad72-4798ac40670b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:11.312394 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e453a40-a2dd-4aa0-ad72-4798ac40670b None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:11.313199 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e453a40-a2dd-4aa0-ad72-4798ac40670b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:11.314967 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e453a40-a2dd-4aa0-ad72-4798ac40670b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:11.325351 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950bc592-f323-44da-b7a7-eb3f0c3cd8e8 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:11.339003 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 376/736] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Jul 25 19:09:11 2022] POST /identity/v3/services => generated 251 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:09:11.341836 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178a1ea-646a-4eb7-a292-80cf8953d904 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:11.342854 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178a1ea-646a-4eb7-a292-80cf8953d904 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:11.343212 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178a1ea-646a-4eb7-a292-80cf8953d904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:11.374545 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3178a1ea-646a-4eb7-a292-80cf8953d904 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:11.376892 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3178a1ea-646a-4eb7-a292-80cf8953d904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:11.393507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8524bfcc-54d8-414e-9887-9bb9a92af756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:11.394237 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8524bfcc-54d8-414e-9887-9bb9a92af756 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:11.394621 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8524bfcc-54d8-414e-9887-9bb9a92af756 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:11.394916 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8524bfcc-54d8-414e-9887-9bb9a92af756 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:11.447843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:11.471798 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:11.507979 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8524bfcc-54d8-414e-9887-9bb9a92af756 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:11.510020 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8524bfcc-54d8-414e-9887-9bb9a92af756 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:11.539622 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 377/737] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:11 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 19:09:11.539622 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3178a1ea-646a-4eb7-a292-80cf8953d904 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:11.703645 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 361/738] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:09:11 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 19:09:11.703645 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f714a-2211-4dbe-99cd-3de14c2c176f None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:11.703645 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f714a-2211-4dbe-99cd-3de14c2c176f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:11.703645 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f714a-2211-4dbe-99cd-3de14c2c176f None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:11.703645 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 362/739] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:11.704886 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39f714a-2211-4dbe-99cd-3de14c2c176f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:11.751658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3178a1ea-646a-4eb7-a292-80cf8953d904 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:11.808421 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:11.812388 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea0bf9-c167-4a9f-a9ea-ff809a3b6c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:11.813507 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea0bf9-c167-4a9f-a9ea-ff809a3b6c3b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:11.813987 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea0bf9-c167-4a9f-a9ea-ff809a3b6c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:11.814535 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea0bf9-c167-4a9f-a9ea-ff809a3b6c3b None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:11.815986 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea0bf9-c167-4a9f-a9ea-ff809a3b6c3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:11.816488 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea0bf9-c167-4a9f-a9ea-ff809a3b6c3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:11.818506 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea0bf9-c167-4a9f-a9ea-ff809a3b6c3b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:11.848492 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 378/740] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:11.850934 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ecfa81-373f-47a0-80ee-be06aa742703 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:11.852143 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ecfa81-373f-47a0-80ee-be06aa742703 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:11.852762 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ecfa81-373f-47a0-80ee-be06aa742703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:11.876948 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-56ecfa81-373f-47a0-80ee-be06aa742703 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:11.878782 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-56ecfa81-373f-47a0-80ee-be06aa742703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:11.884128 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39f714a-2211-4dbe-99cd-3de14c2c176f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:11.913166 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:11.937717 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bbc6df-bac5-490c-8378-e862bc66f70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:11.938493 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bbc6df-bac5-490c-8378-e862bc66f70d None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:11.938782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bbc6df-bac5-490c-8378-e862bc66f70d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:11.939063 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bbc6df-bac5-490c-8378-e862bc66f70d None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:11.939885 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bbc6df-bac5-490c-8378-e862bc66f70d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:11.941313 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bbc6df-bac5-490c-8378-e862bc66f70d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:11.969525 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 363/741] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Jul 25 19:09:11 2022] POST /identity/v3/services => generated 240 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:09:11.970217 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bbc6df-bac5-490c-8378-e862bc66f70d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:12.028482 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 379/742] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:09:12.029599 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ecfa81-373f-47a0-80ee-be06aa742703 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:12.189408 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bbc6df-bac5-490c-8378-e862bc66f70d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:12.221894 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:12.245427 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ecfa81-373f-47a0-80ee-be06aa742703 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:12.249007 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55099d77-44a9-4ad7-8090-bfeadfcdcb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:12.250398 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55099d77-44a9-4ad7-8090-bfeadfcdcb5d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:12.250899 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55099d77-44a9-4ad7-8090-bfeadfcdcb5d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:12.251396 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55099d77-44a9-4ad7-8090-bfeadfcdcb5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:12.273969 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55099d77-44a9-4ad7-8090-bfeadfcdcb5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:12.276967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55099d77-44a9-4ad7-8090-bfeadfcdcb5d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:12.287704 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc67d7-a6d0-4f55-89dd-394335b0bb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:12.288559 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc67d7-a6d0-4f55-89dd-394335b0bb2b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:12.288939 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc67d7-a6d0-4f55-89dd-394335b0bb2b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:12.289421 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc67d7-a6d0-4f55-89dd-394335b0bb2b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:12.301505 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dc67d7-a6d0-4f55-89dd-394335b0bb2b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:12.303123 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dc67d7-a6d0-4f55-89dd-394335b0bb2b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:12.314841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-55099d77-44a9-4ad7-8090-bfeadfcdcb5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:12.314841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:12.314841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:12.314841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:12.314841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:12.314841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:12.314841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:12.314841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:12.314841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:12.314841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:12.314841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:12.314841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:12.314841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:09:12.314841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:09:12.314841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:09:12.316677 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:09:12.316677 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:12.316677 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:12.316677 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:12.316677 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:12.316677 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:12.316677 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:12.316677 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:12.316677 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:12.316677 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:12.316677 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:12.316677 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:12.316677 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:12.316677 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:12.316677 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:12.316677 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:12.316677 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:12.318462 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:09:12.318462 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:09:12.318462 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:12.318462 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:12.318462 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 364/743] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:09:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:12.319752 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55099d77-44a9-4ad7-8090-bfeadfcdcb5d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:12.327846 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-96dc67d7-a6d0-4f55-89dd-394335b0bb2b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 19:09:12.327846 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:12.327846 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:12.327846 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:12.327846 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:12.327846 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:12.327846 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:12.327846 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:12.327846 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:12.327846 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:12.327846 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:12.327846 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:12.327846 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:09:12.327846 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:09:12.327846 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:09:12.327846 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:09:12.331726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:12.331726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:12.331726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:09:12.331726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:09:12.331726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:09:12.331726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:09:12.331726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:12.331726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:12.331726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:12.331726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:12.331726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:12.331726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:12.331726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:12.331726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:12.331726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:12.331726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:12.331726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:12.331726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:12.335431 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:12.335431 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:12.335431 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:09:12.335431 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:09:12.335431 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:09:12.335431 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:09:12.335431 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:09:12.335431 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:09:12.335431 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 19:09:12.335431 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:12.335431 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 380/744] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Jul 25 19:09:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:12.335431 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc67d7-a6d0-4f55-89dd-394335b0bb2b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:12.384325 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:12.537605 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55099d77-44a9-4ad7-8090-bfeadfcdcb5d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:12.565297 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc67d7-a6d0-4f55-89dd-394335b0bb2b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:12.612308 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c138a-b9e8-4c7e-bf77-4086890b7175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:12.614159 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c138a-b9e8-4c7e-bf77-4086890b7175 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:12.614692 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c138a-b9e8-4c7e-bf77-4086890b7175 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:12.615222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c138a-b9e8-4c7e-bf77-4086890b7175 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:12.641823 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c138a-b9e8-4c7e-bf77-4086890b7175 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:09:12.658298 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-523c138a-b9e8-4c7e-bf77-4086890b7175 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:09:12.658298 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:12.658298 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:12.658298 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:12.658298 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:12.658298 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:12.658298 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:12.658298 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:12.658298 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:12.658298 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:12.658298 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:12.658298 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:12.658298 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:09:12.658298 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:09:12.658298 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:09:12.658298 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:09:12.659475 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:09:12.659475 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:09:12.659475 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:09:12.659475 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:09:12.659475 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:12.659475 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:12.659475 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:12.659475 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:12.659475 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:12.659475 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:12.659475 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:12.659475 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:12.659475 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:12.659475 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:12.659475 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:12.659475 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:12.661198 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:12.661198 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:12.661198 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:12.661198 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:12.661198 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:09:12.661198 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:09:12.661198 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:09:12.661198 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:09:12.661198 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:09:12.661198 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:09:12.661198 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:09:12.661198 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:12.662007 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 365/745] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Jul 25 19:09:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:12.667977 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c138a-b9e8-4c7e-bf77-4086890b7175 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:12.670841 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236feaa-c6f9-4f10-b907-421de76c446c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:12.672535 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236feaa-c6f9-4f10-b907-421de76c446c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:12.673410 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236feaa-c6f9-4f10-b907-421de76c446c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:12.674158 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236feaa-c6f9-4f10-b907-421de76c446c None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:12.676239 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b236feaa-c6f9-4f10-b907-421de76c446c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:12.677230 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b236feaa-c6f9-4f10-b907-421de76c446c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:12.680101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b236feaa-c6f9-4f10-b907-421de76c446c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:12.716508 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 381/746] 10.222.0.18 () {56 vars in 1135 bytes} [Mon Jul 25 19:09:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:12.716508 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236feaa-c6f9-4f10-b907-421de76c446c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:12.759263 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:12.855974 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:12.902092 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236feaa-c6f9-4f10-b907-421de76c446c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:12.903913 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c138a-b9e8-4c7e-bf77-4086890b7175 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:12.977398 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc26c8d-b912-406e-bec3-79cda3348047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:12.979017 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc26c8d-b912-406e-bec3-79cda3348047 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:12.980469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc26c8d-b912-406e-bec3-79cda3348047 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:12.981190 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc26c8d-b912-406e-bec3-79cda3348047 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:12.983127 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc26c8d-b912-406e-bec3-79cda3348047 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:12.984079 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc26c8d-b912-406e-bec3-79cda3348047 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:12.987081 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc26c8d-b912-406e-bec3-79cda3348047 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:12.991475 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8c002-f09e-4980-994f-f43a4502bf70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:12.992698 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8c002-f09e-4980-994f-f43a4502bf70 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:12.994100 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8c002-f09e-4980-994f-f43a4502bf70 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:12.994604 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8c002-f09e-4980-994f-f43a4502bf70 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:12.997729 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b8c002-f09e-4980-994f-f43a4502bf70 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:12.998314 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b8c002-f09e-4980-994f-f43a4502bf70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:13.001059 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b8c002-f09e-4980-994f-f43a4502bf70 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:13.016528 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 366/747] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:09:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:13.018626 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc26c8d-b912-406e-bec3-79cda3348047 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:13.031552 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 382/748] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:09:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:13.031552 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8c002-f09e-4980-994f-f43a4502bf70 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:13.251313 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:13.264365 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc26c8d-b912-406e-bec3-79cda3348047 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:13.288686 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8c002-f09e-4980-994f-f43a4502bf70 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:13.356355 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03636440-c290-4d76-99a9-0f2c14d6b7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:13.359041 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03636440-c290-4d76-99a9-0f2c14d6b7ab None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:13.360042 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03636440-c290-4d76-99a9-0f2c14d6b7ab None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:13.360813 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03636440-c290-4d76-99a9-0f2c14d6b7ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:13.365647 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0175af03-9f80-46fd-a067-558b1de71d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:13.367391 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175af03-9f80-46fd-a067-558b1de71d4f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:13.368049 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175af03-9f80-46fd-a067-558b1de71d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:13.368643 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175af03-9f80-46fd-a067-558b1de71d4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:13.386147 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0175af03-9f80-46fd-a067-558b1de71d4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:13.390293 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0175af03-9f80-46fd-a067-558b1de71d4f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:13.396220 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:13.428476 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-0175af03-9f80-46fd-a067-558b1de71d4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:09:13.428476 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:13.428476 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:13.428476 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:13.428476 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:13.428476 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:13.428476 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:13.428476 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:13.428476 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:13.428476 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:13.428476 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:13.428476 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:13.428476 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:09:13.428476 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:09:13.428476 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:09:13.430264 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:09:13.430264 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:13.430264 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:13.430264 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:13.430264 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:13.430264 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:13.430264 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:13.430264 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:13.430264 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:13.430264 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:13.430264 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:13.430264 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:13.430264 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:13.430264 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:13.430264 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:13.430264 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:13.430264 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:13.433075 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:09:13.433075 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:09:13.433075 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:09:13.433075 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:09:13.433075 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:09:13.433075 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:09:13.433075 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:09:13.433075 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:13.433075 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 383/749] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:09:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:13.433075 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0175af03-9f80-46fd-a067-558b1de71d4f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:13.544681 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03636440-c290-4d76-99a9-0f2c14d6b7ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:13.547823 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03636440-c290-4d76-99a9-0f2c14d6b7ab None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:13.609182 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0175af03-9f80-46fd-a067-558b1de71d4f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:13.670527 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ec1b1f-9435-4ff8-8d97-8c95fab89649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:13.672356 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec1b1f-9435-4ff8-8d97-8c95fab89649 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:13.673082 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec1b1f-9435-4ff8-8d97-8c95fab89649 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:13.673586 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec1b1f-9435-4ff8-8d97-8c95fab89649 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:13.692818 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ec1b1f-9435-4ff8-8d97-8c95fab89649 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:13.694451 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ec1b1f-9435-4ff8-8d97-8c95fab89649 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:13.711148 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:13.728776 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d1ec1b1f-9435-4ff8-8d97-8c95fab89649 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:13.728776 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:13.728776 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:13.728776 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:13.728776 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:13.728776 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:13.728776 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:13.728776 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:13.728776 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:13.728776 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:13.728776 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:13.728776 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:13.728776 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:09:13.728776 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:09:13.728776 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:09:13.731056 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:09:13.731056 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:13.731056 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:13.731056 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:13.731056 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:13.731056 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:13.731056 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:13.731056 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:13.731056 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:13.731056 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:13.731056 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:13.731056 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:13.731056 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:13.731056 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:13.731056 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:13.731056 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:13.731056 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:13.733920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:09:13.733920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:09:13.733920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:13.733920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:13.734434 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 367/750] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:09:13 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 714 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 19:09:13.734585 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 384/751] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:09:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:13.734728 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03636440-c290-4d76-99a9-0f2c14d6b7ab None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:13.740683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ec1b1f-9435-4ff8-8d97-8c95fab89649 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:13.821541 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:13.890806 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ec1b1f-9435-4ff8-8d97-8c95fab89649 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:13.920707 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03636440-c290-4d76-99a9-0f2c14d6b7ab None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:13.953389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c00a068-34b4-4935-94cc-9853bac2e7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:13.954344 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c00a068-34b4-4935-94cc-9853bac2e7e3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:13.954764 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c00a068-34b4-4935-94cc-9853bac2e7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:13.955120 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c00a068-34b4-4935-94cc-9853bac2e7e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:13.970503 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c00a068-34b4-4935-94cc-9853bac2e7e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:13.972161 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c00a068-34b4-4935-94cc-9853bac2e7e3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:13.987948 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa39552-6675-473f-942d-691cef2deca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:13.988632 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa39552-6675-473f-942d-691cef2deca7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:13.988917 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa39552-6675-473f-942d-691cef2deca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:13.989198 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa39552-6675-473f-942d-691cef2deca7 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:13.990028 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa39552-6675-473f-942d-691cef2deca7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:13.990346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa39552-6675-473f-942d-691cef2deca7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:13.991931 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa39552-6675-473f-942d-691cef2deca7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:13.996480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-6c00a068-34b4-4935-94cc-9853bac2e7e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:13.996480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:13.996480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:13.996480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:13.996480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:13.996480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:13.996480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:13.996480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:13.996480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:13.996480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:13.996480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:13.996480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:13.996480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:09:13.996480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:09:13.996480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:09:13.999669 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:09:13.999669 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:13.999669 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:13.999669 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:13.999669 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:13.999669 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:13.999669 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:13.999669 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:13.999669 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:13.999669 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:13.999669 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:13.999669 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:13.999669 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:13.999669 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:13.999669 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:13.999669 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:13.999669 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:14.008525 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:09:14.008525 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:09:14.008525 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:14.008525 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:14.008525 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 385/752] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:09:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:14.008525 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c00a068-34b4-4935-94cc-9853bac2e7e3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:14.009865 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 368/753] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Jul 25 19:09:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:14.009865 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa39552-6675-473f-942d-691cef2deca7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:14.086508 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:14.148595 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa39552-6675-473f-942d-691cef2deca7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:14.149310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c00a068-34b4-4935-94cc-9853bac2e7e3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:14.152996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:14.197626 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fefd85-9fb1-4641-8c06-77999d1c542f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:14.198778 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fefd85-9fb1-4641-8c06-77999d1c542f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:14.199295 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fefd85-9fb1-4641-8c06-77999d1c542f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:14.199615 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fefd85-9fb1-4641-8c06-77999d1c542f None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:14.200455 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fefd85-9fb1-4641-8c06-77999d1c542f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:14.200778 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fefd85-9fb1-4641-8c06-77999d1c542f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:14.201999 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4fc9b6-4b7d-4a0f-aca7-2a8ecd01e1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:14.202348 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fefd85-9fb1-4641-8c06-77999d1c542f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:14.203031 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4fc9b6-4b7d-4a0f-aca7-2a8ecd01e1b6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:14.203605 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4fc9b6-4b7d-4a0f-aca7-2a8ecd01e1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:14.204161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4fc9b6-4b7d-4a0f-aca7-2a8ecd01e1b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:14.205161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4fc9b6-4b7d-4a0f-aca7-2a8ecd01e1b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:14.205623 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4fc9b6-4b7d-4a0f-aca7-2a8ecd01e1b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:14.207764 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4fc9b6-4b7d-4a0f-aca7-2a8ecd01e1b6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:14.224573 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 386/754] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:09:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:14.225878 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 369/755] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:09:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:14.226172 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fefd85-9fb1-4641-8c06-77999d1c542f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:14.227063 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4fc9b6-4b7d-4a0f-aca7-2a8ecd01e1b6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:14.367532 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4fc9b6-4b7d-4a0f-aca7-2a8ecd01e1b6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:14.389905 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:14.418116 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b72687-0261-40d2-9930-0a987e70902d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:14.419636 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b72687-0261-40d2-9930-0a987e70902d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:14.419963 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b72687-0261-40d2-9930-0a987e70902d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:14.420245 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b72687-0261-40d2-9930-0a987e70902d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:14.430575 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b72687-0261-40d2-9930-0a987e70902d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:14.432217 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b72687-0261-40d2-9930-0a987e70902d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:14.450785 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-a5b72687-0261-40d2-9930-0a987e70902d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 19:09:14.450785 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:14.450785 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:14.450785 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:14.450785 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:14.450785 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:14.450785 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:14.450785 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:14.450785 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:14.450785 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:14.450785 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:14.450785 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:14.450785 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:09:14.450785 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:09:14.450785 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:09:14.450785 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:09:14.452232 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:14.452232 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:14.452232 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:09:14.452232 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:09:14.452232 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:09:14.452232 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:09:14.452232 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:14.452232 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:14.452232 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:14.452232 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:14.452232 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:14.452232 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:14.452232 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:14.452232 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:14.452232 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:14.452232 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:14.452232 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:14.452232 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:14.453402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:14.453402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:14.453402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:09:14.453402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:09:14.453402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:09:14.453402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:09:14.453402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:09:14.453402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:09:14.453402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 19:09:14.453402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:14.453402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 370/756] 10.222.0.18 () {56 vars in 1093 bytes} [Mon Jul 25 19:09:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:14.454076 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb903dc-8d3d-4f9d-990c-939d67a1a53d None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:14.454658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb903dc-8d3d-4f9d-990c-939d67a1a53d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:14.454658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb903dc-8d3d-4f9d-990c-939d67a1a53d None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:14.460299 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 371/757] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:14.460299 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb903dc-8d3d-4f9d-990c-939d67a1a53d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:14.500173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fefd85-9fb1-4641-8c06-77999d1c542f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:14.546014 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f78a6d4-042c-40e5-b97b-ebdd26769abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:14.546787 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78a6d4-042c-40e5-b97b-ebdd26769abb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:14.547080 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78a6d4-042c-40e5-b97b-ebdd26769abb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:14.547368 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78a6d4-042c-40e5-b97b-ebdd26769abb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:14.548442 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f78a6d4-042c-40e5-b97b-ebdd26769abb None admin] RBAC: Adding query filter params (role.id=23718d83b0864b16b0071e8ffe0b5861, scope.project.id=191069caccd04233b012585e29cfa18b, user.id=1c0772205ae24713b8901666a87018b2) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:14.548940 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f78a6d4-042c-40e5-b97b-ebdd26769abb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:14.550304 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f78a6d4-042c-40e5-b97b-ebdd26769abb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:14.567551 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 387/758] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:09:14 2022] GET /identity/v3/role_assignments?user.id=1c0772205ae24713b8901666a87018b2&scope.project.id=191069caccd04233b012585e29cfa18b&role.id=23718d83b0864b16b0071e8ffe0b5861 => generated 259 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:14.567551 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f78a6d4-042c-40e5-b97b-ebdd26769abb None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:14.602614 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb903dc-8d3d-4f9d-990c-939d67a1a53d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:14.619820 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:14.654605 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6fbc8-970e-49c1-b86b-e3163be2edfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:14.655445 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6fbc8-970e-49c1-b86b-e3163be2edfa None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:14.655937 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6fbc8-970e-49c1-b86b-e3163be2edfa None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:14.656193 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6fbc8-970e-49c1-b86b-e3163be2edfa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:14.665032 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6fbc8-970e-49c1-b86b-e3163be2edfa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:14.666866 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6fbc8-970e-49c1-b86b-e3163be2edfa None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:14.686335 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-aca6fbc8-970e-49c1-b86b-e3163be2edfa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:09:14.686335 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:14.686335 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:14.686335 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:14.686335 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:14.686335 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:14.686335 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:14.686335 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:14.686335 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:14.686335 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:14.686335 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:14.686335 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:14.686335 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:09:14.686335 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:09:14.686335 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:09:14.688161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:09:14.688161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:14.688161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:14.688161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:14.688161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:14.688161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:14.688161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:14.688161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:14.688161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:14.688161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:14.688161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:14.688161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:14.688161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:14.688161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:14.688161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:14.688161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:14.688161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:14.689992 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:09:14.689992 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:09:14.689992 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:09:14.689992 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:09:14.689992 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:09:14.689992 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:09:14.689992 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:09:14.689992 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:14.689992 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 372/759] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:09:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:14.689992 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2d93c-8797-4a51-b716-0192b470ac70 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:14.689992 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2d93c-8797-4a51-b716-0192b470ac70 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:14.691203 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2d93c-8797-4a51-b716-0192b470ac70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:14.723931 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-efe2d93c-8797-4a51-b716-0192b470ac70 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:14.725228 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-efe2d93c-8797-4a51-b716-0192b470ac70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:14.766536 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f78a6d4-042c-40e5-b97b-ebdd26769abb None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:14.769830 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:14.825668 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 373/760] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:09:14.827329 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe2d93c-8797-4a51-b716-0192b470ac70 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:14.835469 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc14b574-20d9-4e9c-950d-d1996889226d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:14.836372 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14b574-20d9-4e9c-950d-d1996889226d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:14.836660 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14b574-20d9-4e9c-950d-d1996889226d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:14.836939 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14b574-20d9-4e9c-950d-d1996889226d None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:14.837825 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc14b574-20d9-4e9c-950d-d1996889226d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:14.838190 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc14b574-20d9-4e9c-950d-d1996889226d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:14.840222 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc14b574-20d9-4e9c-950d-d1996889226d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:14.865561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 388/761] 10.222.0.18 () {56 vars in 1133 bytes} [Mon Jul 25 19:09:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:14.866644 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc14b574-20d9-4e9c-950d-d1996889226d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:15.006908 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:15.053100 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc14b574-20d9-4e9c-950d-d1996889226d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:15.060893 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe2d93c-8797-4a51-b716-0192b470ac70 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:15.126632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a262a8-6054-4ee6-abb7-77b9cff342dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:15.128653 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a262a8-6054-4ee6-abb7-77b9cff342dd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:15.129052 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a262a8-6054-4ee6-abb7-77b9cff342dd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:15.129314 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a262a8-6054-4ee6-abb7-77b9cff342dd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:15.131202 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a262a8-6054-4ee6-abb7-77b9cff342dd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:15.132997 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a262a8-6054-4ee6-abb7-77b9cff342dd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:15.141445 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30af84a7-da58-42e2-8fda-0c99bc3f66c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:15.143305 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af84a7-da58-42e2-8fda-0c99bc3f66c8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:15.144164 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af84a7-da58-42e2-8fda-0c99bc3f66c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:15.144894 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af84a7-da58-42e2-8fda-0c99bc3f66c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:15.146999 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30af84a7-da58-42e2-8fda-0c99bc3f66c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:15.147951 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30af84a7-da58-42e2-8fda-0c99bc3f66c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:15.150525 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-c0a262a8-6054-4ee6-abb7-77b9cff342dd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 19:09:15.150525 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:15.150525 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 19:09:15.150525 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 19:09:15.150525 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 19:09:15.150525 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 19:09:15.150525 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 19:09:15.150525 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:15.150525 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 19:09:15.150525 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:15.150525 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 19:09:15.150525 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:15.150525 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:15.150525 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:15.150525 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:15.150525 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:15.152788 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:15.152788 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:15.152788 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:15.152788 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:15.152788 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:15.152788 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:15.152788 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:15.152788 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 19:09:15.152788 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 19:09:15.152788 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 19:09:15.152788 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 19:09:15.152788 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:15.152788 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:15.152788 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:15.152788 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:15.152788 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:15.154413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:15.154413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:15.154413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:15.154413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:15.154413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:15.154413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:15.154413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:15.154413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:15.154413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:15.154413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:15.154413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:15.154413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 19:09:15.154413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:15.154413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 19:09:15.154413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:15.154413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 389/762] 10.222.0.18 () {56 vars in 1129 bytes} [Mon Jul 25 19:09:14 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:15.156177 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a262a8-6054-4ee6-abb7-77b9cff342dd None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:15.159748 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30af84a7-da58-42e2-8fda-0c99bc3f66c8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:15.190001 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 374/763] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Jul 25 19:09:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:15.190975 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30af84a7-da58-42e2-8fda-0c99bc3f66c8 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:15.235675 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:15.372627 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a262a8-6054-4ee6-abb7-77b9cff342dd None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:15.378263 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30af84a7-da58-42e2-8fda-0c99bc3f66c8 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:15.439495 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f1ecf-239e-4b51-96d2-73cb73a2d4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:15.440378 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f1ecf-239e-4b51-96d2-73cb73a2d4cf None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:15.440729 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f1ecf-239e-4b51-96d2-73cb73a2d4cf None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:15.441038 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f1ecf-239e-4b51-96d2-73cb73a2d4cf None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:15.441850 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f1ecf-239e-4b51-96d2-73cb73a2d4cf None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:15.442195 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f1ecf-239e-4b51-96d2-73cb73a2d4cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:15.443884 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f1ecf-239e-4b51-96d2-73cb73a2d4cf None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:15.446169 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb9453-9761-4d75-8485-562d35feaad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:15.447672 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb9453-9761-4d75-8485-562d35feaad9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:15.448189 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb9453-9761-4d75-8485-562d35feaad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:15.448748 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb9453-9761-4d75-8485-562d35feaad9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:15.458560 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 375/764] 10.222.0.18 () {56 vars in 1124 bytes} [Mon Jul 25 19:09:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:15.458560 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f1ecf-239e-4b51-96d2-73cb73a2d4cf None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:15.474429 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbb9453-9761-4d75-8485-562d35feaad9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:15.476333 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbb9453-9761-4d75-8485-562d35feaad9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:15.501282 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-5fbb9453-9761-4d75-8485-562d35feaad9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:15.501282 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:15.501282 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:15.501282 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:15.501282 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:15.501282 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:15.501282 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:15.501282 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:15.501282 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:15.501282 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:15.501282 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:15.501282 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:15.501282 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:09:15.501282 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:09:15.501282 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:09:15.502920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:09:15.502920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:15.502920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:15.502920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:15.502920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:15.502920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:15.502920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:15.502920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:15.502920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:15.502920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:15.502920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:15.502920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:15.502920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:15.502920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:15.502920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:15.502920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:15.502920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:15.505126 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:09:15.505126 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:09:15.505126 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:15.505126 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:15.505126 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 390/765] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:09:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:15.505126 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc52acdc-8ff5-4e56-9548-aa5e7294425b None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:15.505126 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc52acdc-8ff5-4e56-9548-aa5e7294425b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:15.505126 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc52acdc-8ff5-4e56-9548-aa5e7294425b None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:15.507035 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 391/766] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:15.511727 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc52acdc-8ff5-4e56-9548-aa5e7294425b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:15.516058 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:15.659755 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:15.685345 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f1ecf-239e-4b51-96d2-73cb73a2d4cf None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:15.689588 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc52acdc-8ff5-4e56-9548-aa5e7294425b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:15.746429 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb5f2ac-7f64-4e11-8146-774216f36827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:15.747363 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb5f2ac-7f64-4e11-8146-774216f36827 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:15.747842 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb5f2ac-7f64-4e11-8146-774216f36827 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:15.748125 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb5f2ac-7f64-4e11-8146-774216f36827 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:15.749001 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb5f2ac-7f64-4e11-8146-774216f36827 None admin] RBAC: Adding query filter params (role.id=ed7fec18968241efa76a0ee769d96da9, scope.project.id=191069caccd04233b012585e29cfa18b, user.id=feafcf1b5a9844fe9738823fddd07e0e) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:15.749321 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb5f2ac-7f64-4e11-8146-774216f36827 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:15.751761 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb5f2ac-7f64-4e11-8146-774216f36827 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:15.767834 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 376/767] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:09:15 2022] GET /identity/v3/role_assignments?user.id=feafcf1b5a9844fe9738823fddd07e0e&scope.project.id=191069caccd04233b012585e29cfa18b&role.id=ed7fec18968241efa76a0ee769d96da9 => generated 259 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:15.768051 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14829259-136e-42ff-84a8-aa1904c7d1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:15.768167 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb5f2ac-7f64-4e11-8146-774216f36827 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:15.768255 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14829259-136e-42ff-84a8-aa1904c7d1ac None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:15.768255 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14829259-136e-42ff-84a8-aa1904c7d1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:15.768255 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14829259-136e-42ff-84a8-aa1904c7d1ac None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:15.768255 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14829259-136e-42ff-84a8-aa1904c7d1ac None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:15.768255 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14829259-136e-42ff-84a8-aa1904c7d1ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:15.768255 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14829259-136e-42ff-84a8-aa1904c7d1ac None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:15.780096 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 392/768] 10.222.0.18 () {56 vars in 1124 bytes} [Mon Jul 25 19:09:15 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:15.782688 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622dec90-21ef-4380-93a2-3efdb51c80ff None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:15.783121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622dec90-21ef-4380-93a2-3efdb51c80ff None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:15.784563 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622dec90-21ef-4380-93a2-3efdb51c80ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:15.818979 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-622dec90-21ef-4380-93a2-3efdb51c80ff None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:15.820733 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-622dec90-21ef-4380-93a2-3efdb51c80ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:15.958840 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:15.971906 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb5f2ac-7f64-4e11-8146-774216f36827 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:15.983560 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 393/769] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:09:15.983560 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622dec90-21ef-4380-93a2-3efdb51c80ff None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:16.056505 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607af7ed-755a-49f2-8b23-ac0510021e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:16.057749 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607af7ed-755a-49f2-8b23-ac0510021e2a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:16.059545 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607af7ed-755a-49f2-8b23-ac0510021e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:16.060087 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607af7ed-755a-49f2-8b23-ac0510021e2a None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:16.061706 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607af7ed-755a-49f2-8b23-ac0510021e2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:16.062274 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607af7ed-755a-49f2-8b23-ac0510021e2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:16.065249 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607af7ed-755a-49f2-8b23-ac0510021e2a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:16.093534 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 377/770] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:09:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:16.095010 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607af7ed-755a-49f2-8b23-ac0510021e2a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:16.132105 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:16.162566 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622dec90-21ef-4380-93a2-3efdb51c80ff None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:16.218433 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863c7be-b0f5-42f0-8fe5-33f83ac2bcd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:16.220107 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863c7be-b0f5-42f0-8fe5-33f83ac2bcd9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:16.220578 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863c7be-b0f5-42f0-8fe5-33f83ac2bcd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:16.221296 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863c7be-b0f5-42f0-8fe5-33f83ac2bcd9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:16.222426 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9863c7be-b0f5-42f0-8fe5-33f83ac2bcd9 None admin] RBAC: Adding query filter params (interface=public, service_id=e413f4ff38014d6f9bb1cf892f9fcc06, region_id=RegionOne) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:16.223205 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9863c7be-b0f5-42f0-8fe5-33f83ac2bcd9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:16.225443 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9863c7be-b0f5-42f0-8fe5-33f83ac2bcd9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:16.238419 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 394/771] 10.222.0.18 () {56 vars in 1251 bytes} [Mon Jul 25 19:09:15 2022] GET /identity/v3/endpoints?service_id=e413f4ff38014d6f9bb1cf892f9fcc06&interface=public®ion_id=RegionOne => generated 194 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:16.240284 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863c7be-b0f5-42f0-8fe5-33f83ac2bcd9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:16.312502 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607af7ed-755a-49f2-8b23-ac0510021e2a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:16.352150 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:16.383835 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23693b55-5f52-49c6-b98c-086c2397ab65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:16.385113 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23693b55-5f52-49c6-b98c-086c2397ab65 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:16.385626 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23693b55-5f52-49c6-b98c-086c2397ab65 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:16.386599 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23693b55-5f52-49c6-b98c-086c2397ab65 None admin] PATH_INFO: `/v3/services/network` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:16.388186 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23693b55-5f52-49c6-b98c-086c2397ab65 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:16.390543 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23693b55-5f52-49c6-b98c-086c2397ab65 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:16.406807 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863c7be-b0f5-42f0-8fe5-33f83ac2bcd9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:16.414170 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-23693b55-5f52-49c6-b98c-086c2397ab65 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 19:09:16.414170 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:16.414170 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 19:09:16.414170 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 19:09:16.414170 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 19:09:16.414170 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 19:09:16.414170 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 19:09:16.414170 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:16.414170 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 19:09:16.414170 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:16.414170 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 19:09:16.414170 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:16.414170 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:16.414170 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:16.414170 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:16.414170 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:16.414170 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:16.415288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:16.415288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:16.415288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:16.415288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:16.415288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:16.415288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:16.415288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 19:09:16.415288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 19:09:16.415288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 19:09:16.415288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 19:09:16.415288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:16.415288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:16.415288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:16.415288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:16.415288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:16.415288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:16.417131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:16.417131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:16.417131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:16.417131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:16.417131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:16.417131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:16.417131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:16.417131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:16.417131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:16.417131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:16.417131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 19:09:16.417131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:16.417131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 19:09:16.417131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:16.417131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 378/772] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:09:16 2022] GET /identity/v3/services/network => generated 88 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:16.418581 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23693b55-5f52-49c6-b98c-086c2397ab65 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:16.471490 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7d355-e0b6-42a8-bf02-d5b9f68190ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:16.473299 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7d355-e0b6-42a8-bf02-d5b9f68190ac None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:16.473875 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7d355-e0b6-42a8-bf02-d5b9f68190ac None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:16.474918 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7d355-e0b6-42a8-bf02-d5b9f68190ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:16.497388 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7d355-e0b6-42a8-bf02-d5b9f68190ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:16.500837 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7d355-e0b6-42a8-bf02-d5b9f68190ac None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:16.539139 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-5ae7d355-e0b6-42a8-bf02-d5b9f68190ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:09:16.539139 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:16.539139 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:16.539139 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:16.539139 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:16.539139 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:16.539139 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:16.539139 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:16.539139 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:16.539139 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:16.539139 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:16.539139 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:16.539139 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:09:16.539139 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:09:16.539139 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:09:16.540908 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:09:16.540908 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:16.540908 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:16.540908 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:16.540908 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:16.540908 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:16.540908 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:16.540908 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:16.540908 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:16.540908 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:16.540908 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:16.540908 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:16.540908 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:16.540908 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:16.540908 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:16.540908 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:16.540908 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:16.542775 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:09:16.542775 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:09:16.542775 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:09:16.542775 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:09:16.542775 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:09:16.542775 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:09:16.542775 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:09:16.542775 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:16.547750 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 395/773] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:09:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:16.550304 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7d355-e0b6-42a8-bf02-d5b9f68190ac None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:16.595464 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:16.642845 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23693b55-5f52-49c6-b98c-086c2397ab65 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:16.720195 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98717dc6-5a40-4bbb-ad81-b84146e47037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:16.722415 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98717dc6-5a40-4bbb-ad81-b84146e47037 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:16.724034 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98717dc6-5a40-4bbb-ad81-b84146e47037 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:16.724847 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98717dc6-5a40-4bbb-ad81-b84146e47037 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:16.726891 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98717dc6-5a40-4bbb-ad81-b84146e47037 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:16.731763 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98717dc6-5a40-4bbb-ad81-b84146e47037 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:16.740460 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 379/774] 10.222.0.18 () {56 vars in 1087 bytes} [Mon Jul 25 19:09:16 2022] GET /identity/v3/services => generated 772 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:16.740775 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7d355-e0b6-42a8-bf02-d5b9f68190ac None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:16.742458 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98717dc6-5a40-4bbb-ad81-b84146e47037 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:16.818731 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:16.858238 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186284cb-075d-44a8-bff4-fe5481427623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:16.859247 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186284cb-075d-44a8-bff4-fe5481427623 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:16.860114 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186284cb-075d-44a8-bff4-fe5481427623 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:16.860545 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186284cb-075d-44a8-bff4-fe5481427623 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:16.863709 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186284cb-075d-44a8-bff4-fe5481427623 None admin] RBAC: Adding query filter params (name=network) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:16.864162 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186284cb-075d-44a8-bff4-fe5481427623 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:16.866748 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186284cb-075d-44a8-bff4-fe5481427623 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:16.877440 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 396/775] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Jul 25 19:09:16 2022] GET /identity/v3/services?name=network => generated 124 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:16.884304 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186284cb-075d-44a8-bff4-fe5481427623 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:17.033618 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186284cb-075d-44a8-bff4-fe5481427623 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:17.053034 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98717dc6-5a40-4bbb-ad81-b84146e47037 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:17.078897 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22248a-652a-476c-a1f1-15d3f311cbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:17.080150 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22248a-652a-476c-a1f1-15d3f311cbf0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:17.080680 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22248a-652a-476c-a1f1-15d3f311cbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:17.081270 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22248a-652a-476c-a1f1-15d3f311cbf0 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:17.082295 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22248a-652a-476c-a1f1-15d3f311cbf0 None admin] RBAC: Adding query filter params (type=network) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:17.082834 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22248a-652a-476c-a1f1-15d3f311cbf0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:17.084954 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22248a-652a-476c-a1f1-15d3f311cbf0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:17.098607 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 397/776] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Jul 25 19:09:16 2022] GET /identity/v3/services?type=network => generated 350 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:17.109917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22248a-652a-476c-a1f1-15d3f311cbf0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:17.135782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf5688d-148e-49f3-a64b-7f7485d28985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:17.136826 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5688d-148e-49f3-a64b-7f7485d28985 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:17.137247 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5688d-148e-49f3-a64b-7f7485d28985 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:17.137660 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5688d-148e-49f3-a64b-7f7485d28985 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:17.139118 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf5688d-148e-49f3-a64b-7f7485d28985 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:17.139627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf5688d-148e-49f3-a64b-7f7485d28985 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:17.142193 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf5688d-148e-49f3-a64b-7f7485d28985 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:17.183631 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 380/777] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Jul 25 19:09:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 441 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:17.193459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf5688d-148e-49f3-a64b-7f7485d28985 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:17.198288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:17.257496 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:17.407677 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22248a-652a-476c-a1f1-15d3f311cbf0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:17.516928 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb844d3-a902-4342-9feb-6ffa9baa7da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:17.517937 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb844d3-a902-4342-9feb-6ffa9baa7da1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:17.518358 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb844d3-a902-4342-9feb-6ffa9baa7da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:17.518839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb844d3-a902-4342-9feb-6ffa9baa7da1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:17.520448 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb844d3-a902-4342-9feb-6ffa9baa7da1 None admin] RBAC: Adding query filter params (interface=public, service_id=15b735f893d14f4c805182859667cdae, region_id=RegionOne) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:17.520990 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb844d3-a902-4342-9feb-6ffa9baa7da1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:17.523644 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb844d3-a902-4342-9feb-6ffa9baa7da1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:17.547385 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 398/778] 10.222.0.18 () {56 vars in 1251 bytes} [Mon Jul 25 19:09:17 2022] GET /identity/v3/endpoints?service_id=15b735f893d14f4c805182859667cdae&interface=public®ion_id=RegionOne => generated 194 bytes in 438 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:17.557020 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb844d3-a902-4342-9feb-6ffa9baa7da1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:17.706145 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf5688d-148e-49f3-a64b-7f7485d28985 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:17.826146 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb844d3-a902-4342-9feb-6ffa9baa7da1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:17.840403 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99411eef-207a-44c2-b52a-3dfce0bd4d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:17.841918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99411eef-207a-44c2-b52a-3dfce0bd4d11 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:17.842233 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99411eef-207a-44c2-b52a-3dfce0bd4d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:17.842587 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99411eef-207a-44c2-b52a-3dfce0bd4d11 None admin] PATH_INFO: `/v3/projects/191069caccd04233b012585e29cfa18b/users/860e7d26ad364795bb25836aadd4ac7b/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:17.849655 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99411eef-207a-44c2-b52a-3dfce0bd4d11 None admin] RBAC: Authorizing `identity:create_grant(project_id=191069caccd04233b012585e29cfa18b, user_id=860e7d26ad364795bb25836aadd4ac7b, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:17.851410 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99411eef-207a-44c2-b52a-3dfce0bd4d11 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:17.857220 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:17.892710 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 381/779] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Jul 25 19:09:17 2022] PUT /identity/v3/projects/191069caccd04233b012585e29cfa18b/users/860e7d26ad364795bb25836aadd4ac7b/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 695 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:09:17.900367 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5815dc-814e-4828-ad4e-9ccedb06005a None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:17.901397 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5815dc-814e-4828-ad4e-9ccedb06005a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:17.903016 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5815dc-814e-4828-ad4e-9ccedb06005a None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:17.906847 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 382/780] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:17 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:17.908327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad48688-02b6-4ece-80a2-a54bde7f1dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:17.910000 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad48688-02b6-4ece-80a2-a54bde7f1dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:17.910886 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad48688-02b6-4ece-80a2-a54bde7f1dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:17.911699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad48688-02b6-4ece-80a2-a54bde7f1dc0 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:17.913358 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad48688-02b6-4ece-80a2-a54bde7f1dc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:17.916289 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad48688-02b6-4ece-80a2-a54bde7f1dc0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:17.926535 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333db81-de8b-499f-ada3-999736d2e56f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:17.927014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333db81-de8b-499f-ada3-999736d2e56f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:17.928056 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333db81-de8b-499f-ada3-999736d2e56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:17.931118 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 399/781] 10.222.0.18 () {56 vars in 1087 bytes} [Mon Jul 25 19:09:17 2022] GET /identity/v3/services => generated 772 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:09:17.966106 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7333db81-de8b-499f-ada3-999736d2e56f None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:17.969101 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7333db81-de8b-499f-ada3-999736d2e56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:18.101667 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:18.127299 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 383/782] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 19:09:18.196338 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad48688-02b6-4ece-80a2-a54bde7f1dc0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:18.320630 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00fe36-4ca8-41a9-9bc9-f88c91b5fbac None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:18.321334 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00fe36-4ca8-41a9-9bc9-f88c91b5fbac None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:18.321334 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00fe36-4ca8-41a9-9bc9-f88c91b5fbac None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:18.323381 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 384/783] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:18.331758 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e2497-adaf-438c-841c-8a69395ff04a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:18.332144 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e2497-adaf-438c-841c-8a69395ff04a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:18.332348 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e2497-adaf-438c-841c-8a69395ff04a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:18.367256 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-9e4e2497-adaf-438c-841c-8a69395ff04a None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:18.370000 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9e4e2497-adaf-438c-841c-8a69395ff04a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:18.438233 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad48688-02b6-4ece-80a2-a54bde7f1dc0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:18.513504 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 385/784] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:09:18.540713 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e2497-adaf-438c-841c-8a69395ff04a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:18.554364 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e248b7f3-cbc2-4cc0-83b1-9f43bd0d97f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:18.555748 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248b7f3-cbc2-4cc0-83b1-9f43bd0d97f2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:18.556235 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248b7f3-cbc2-4cc0-83b1-9f43bd0d97f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:18.556703 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248b7f3-cbc2-4cc0-83b1-9f43bd0d97f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:18.608063 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e248b7f3-cbc2-4cc0-83b1-9f43bd0d97f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:09:18.625143 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e248b7f3-cbc2-4cc0-83b1-9f43bd0d97f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:09:18.625143 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:18.625143 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:18.625143 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:18.625143 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:18.625143 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:18.625143 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:18.625143 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:18.625143 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:18.625143 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:18.625143 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:18.625143 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:18.625143 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:09:18.625143 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:09:18.625143 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:09:18.625143 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:09:18.626748 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:09:18.626748 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:09:18.626748 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:09:18.626748 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:09:18.626748 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:18.626748 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:18.626748 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:18.626748 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:18.626748 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:18.626748 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:18.626748 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:18.626748 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:18.626748 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:18.626748 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:18.626748 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:18.626748 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:18.628347 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:18.628347 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:18.628347 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:18.628347 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:18.628347 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:09:18.628347 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:09:18.628347 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:09:18.628347 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:09:18.628347 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:09:18.628347 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:09:18.628347 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:09:18.628347 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:18.628347 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 400/785] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Jul 25 19:09:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 433 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:18.638640 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e248b7f3-cbc2-4cc0-83b1-9f43bd0d97f2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:18.683053 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e2497-adaf-438c-841c-8a69395ff04a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:18.716935 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:18.740340 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:18.745023 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661182c2-18a5-4553-8935-226f0ff65c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:18.746169 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661182c2-18a5-4553-8935-226f0ff65c4f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:18.746680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661182c2-18a5-4553-8935-226f0ff65c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:18.746980 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661182c2-18a5-4553-8935-226f0ff65c4f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:18.762592 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661182c2-18a5-4553-8935-226f0ff65c4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:09:18.772745 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-661182c2-18a5-4553-8935-226f0ff65c4f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:09:18.772745 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:18.772745 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:18.772745 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:18.772745 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:18.772745 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:18.772745 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:18.772745 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:18.772745 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:18.772745 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:18.772745 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:18.772745 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:18.772745 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:09:18.772745 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:09:18.772745 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:09:18.772745 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:09:18.776395 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:09:18.776395 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:09:18.776395 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:09:18.776395 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:09:18.776395 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:18.776395 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:18.776395 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:18.776395 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:18.776395 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:18.776395 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:18.776395 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:18.776395 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:18.776395 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:18.776395 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:18.776395 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:18.776395 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:18.780142 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:18.780142 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:18.780142 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:18.780142 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:18.780142 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:09:18.780142 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:09:18.780142 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:09:18.780142 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:09:18.780142 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:09:18.780142 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:09:18.780142 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:09:18.780142 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:18.780142 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 386/786] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Jul 25 19:09:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:09:18.791551 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661182c2-18a5-4553-8935-226f0ff65c4f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:18.918073 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e248b7f3-cbc2-4cc0-83b1-9f43bd0d97f2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:18.998378 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f46eec-97c0-42ae-8bc5-b11da03b5faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:18.999175 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f46eec-97c0-42ae-8bc5-b11da03b5faf None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:19.003550 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f46eec-97c0-42ae-8bc5-b11da03b5faf None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:19.003550 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f46eec-97c0-42ae-8bc5-b11da03b5faf None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:19.003550 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f46eec-97c0-42ae-8bc5-b11da03b5faf None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:19.003550 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f46eec-97c0-42ae-8bc5-b11da03b5faf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:19.004523 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f46eec-97c0-42ae-8bc5-b11da03b5faf None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:19.028274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 401/787] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Jul 25 19:09:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:09:19.037308 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f46eec-97c0-42ae-8bc5-b11da03b5faf None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:19.171259 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661182c2-18a5-4553-8935-226f0ff65c4f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:19.273015 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:19.287670 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f46eec-97c0-42ae-8bc5-b11da03b5faf None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:19.319646 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee05d637-89f8-4178-abf6-c01074fd7c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:19.321634 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee05d637-89f8-4178-abf6-c01074fd7c4b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:19.322549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee05d637-89f8-4178-abf6-c01074fd7c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:19.322549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee05d637-89f8-4178-abf6-c01074fd7c4b None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:19.324644 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee05d637-89f8-4178-abf6-c01074fd7c4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:19.325228 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee05d637-89f8-4178-abf6-c01074fd7c4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:19.331565 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee05d637-89f8-4178-abf6-c01074fd7c4b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:19.350879 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 387/788] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:09:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 566 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:09:19.361471 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee05d637-89f8-4178-abf6-c01074fd7c4b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:19.404041 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f29098-f6fa-4368-8215-ea2d924b9e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:19.405506 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f29098-f6fa-4368-8215-ea2d924b9e16 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:19.406014 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f29098-f6fa-4368-8215-ea2d924b9e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:19.406510 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f29098-f6fa-4368-8215-ea2d924b9e16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:19.445266 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f29098-f6fa-4368-8215-ea2d924b9e16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:19.447663 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f29098-f6fa-4368-8215-ea2d924b9e16 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:19.489259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-38f29098-f6fa-4368-8215-ea2d924b9e16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:19.489259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:19.489259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:19.489259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:19.489259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:19.489259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:19.489259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:19.489259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:19.489259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:19.489259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:19.489259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:19.489259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:19.489259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:09:19.489259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:09:19.489259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:09:19.490397 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:09:19.490397 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:19.490397 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:19.490397 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:19.490397 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:19.490397 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:19.490397 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:19.490397 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:19.490397 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:19.490397 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:19.490397 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:19.490397 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:19.490397 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:19.490397 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:19.490397 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:19.490397 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:19.490397 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:19.491327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:09:19.491327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:09:19.491327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:19.491327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:19.492132 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 402/789] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:09:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 456 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:19.502162 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f29098-f6fa-4368-8215-ea2d924b9e16 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:19.508251 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:19.593922 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee05d637-89f8-4178-abf6-c01074fd7c4b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:19.686511 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e9c2ee-444e-4d00-b0ca-98e8b09c8680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:19.687987 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9c2ee-444e-4d00-b0ca-98e8b09c8680 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:19.688757 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9c2ee-444e-4d00-b0ca-98e8b09c8680 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:19.691027 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9c2ee-444e-4d00-b0ca-98e8b09c8680 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:19.719046 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e9c2ee-444e-4d00-b0ca-98e8b09c8680 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:19.720547 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e9c2ee-444e-4d00-b0ca-98e8b09c8680 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:19.762870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-26e9c2ee-444e-4d00-b0ca-98e8b09c8680 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:19.762870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:19.762870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:19.762870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:19.762870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:19.762870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:19.762870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:19.762870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:19.762870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:19.762870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:19.762870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:19.762870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:19.762870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:09:19.762870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:09:19.762870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:09:19.764654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:09:19.764654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:19.764654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:19.764654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:19.764654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:19.764654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:19.764654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:19.764654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:19.764654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:19.764654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:19.764654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:19.764654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:19.764654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:19.764654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:19.764654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:19.764654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:19.764654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:19.766631 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:09:19.766631 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:09:19.766631 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:19.766631 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:19.766631 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 388/790] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:09:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:09:19.773136 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e9c2ee-444e-4d00-b0ca-98e8b09c8680 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:19.775407 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f29098-f6fa-4368-8215-ea2d924b9e16 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:19.829487 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a722e-1d53-45bf-9500-8914ea474914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:19.830613 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a722e-1d53-45bf-9500-8914ea474914 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:19.831133 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a722e-1d53-45bf-9500-8914ea474914 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:19.831681 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a722e-1d53-45bf-9500-8914ea474914 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:19.832772 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41a722e-1d53-45bf-9500-8914ea474914 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:19.833291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41a722e-1d53-45bf-9500-8914ea474914 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:19.835288 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41a722e-1d53-45bf-9500-8914ea474914 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:19.868482 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 403/791] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:09:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:19.877346 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a722e-1d53-45bf-9500-8914ea474914 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:19.904407 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:19.924034 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:19.932408 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e9c2ee-444e-4d00-b0ca-98e8b09c8680 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:19.989240 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac79df97-714c-4834-b2ff-b6ff2e657027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:19.993513 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79df97-714c-4834-b2ff-b6ff2e657027 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:19.994585 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79df97-714c-4834-b2ff-b6ff2e657027 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:19.995676 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79df97-714c-4834-b2ff-b6ff2e657027 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:19.998906 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac79df97-714c-4834-b2ff-b6ff2e657027 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:20.000157 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac79df97-714c-4834-b2ff-b6ff2e657027 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:20.005821 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac79df97-714c-4834-b2ff-b6ff2e657027 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:20.029985 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 389/792] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:09:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:09:20.040766 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac79df97-714c-4834-b2ff-b6ff2e657027 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:20.129368 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a722e-1d53-45bf-9500-8914ea474914 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:20.215112 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5407307-43db-4763-8d19-71879ba9d1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:20.217343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5407307-43db-4763-8d19-71879ba9d1cb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:20.219824 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5407307-43db-4763-8d19-71879ba9d1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:20.220649 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5407307-43db-4763-8d19-71879ba9d1cb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:20.253839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5407307-43db-4763-8d19-71879ba9d1cb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:20.257327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5407307-43db-4763-8d19-71879ba9d1cb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:20.336258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e5407307-43db-4763-8d19-71879ba9d1cb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 19:09:20.336258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:20.336258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:20.336258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:20.336258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:20.336258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:20.336258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:20.336258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:20.336258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:20.336258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:20.336258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:20.336258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:20.336258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:09:20.336258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:09:20.336258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:09:20.336258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:09:20.337999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:20.337999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:20.337999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:09:20.337999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:09:20.337999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:09:20.337999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:09:20.337999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:20.337999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:20.337999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:20.337999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:20.337999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:20.337999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:20.337999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:20.337999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:20.337999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:20.337999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:20.337999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:20.337999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:20.342599 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:20.342599 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:20.342599 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:09:20.342599 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:09:20.342599 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:09:20.342599 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:09:20.342599 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:09:20.342599 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:09:20.342599 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 19:09:20.342599 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:20.347821 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 404/793] 10.222.0.18 () {56 vars in 1117 bytes} [Mon Jul 25 19:09:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 471 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:09:20.360558 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5407307-43db-4763-8d19-71879ba9d1cb None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:20.389506 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac79df97-714c-4834-b2ff-b6ff2e657027 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:20.525286 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:20.531558 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fca34-cbe3-4d05-a2cb-076eb5d28502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:20.532087 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fca34-cbe3-4d05-a2cb-076eb5d28502 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:20.532565 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fca34-cbe3-4d05-a2cb-076eb5d28502 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:20.533043 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fca34-cbe3-4d05-a2cb-076eb5d28502 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:20.570710 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183fca34-cbe3-4d05-a2cb-076eb5d28502 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:20.579959 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183fca34-cbe3-4d05-a2cb-076eb5d28502 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:20.619244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-183fca34-cbe3-4d05-a2cb-076eb5d28502 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 19:09:20.619244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:20.619244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:20.619244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:20.619244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:20.619244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:20.619244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:20.619244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:20.619244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:20.619244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:20.619244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:20.619244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:20.619244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:09:20.619244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:09:20.619244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:09:20.619244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:09:20.621205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:20.621205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:20.621205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:09:20.621205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:09:20.621205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:09:20.621205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:09:20.621205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:20.621205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:20.621205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:20.621205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:20.621205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:20.621205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:20.621205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:20.621205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:20.621205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:20.621205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:20.621205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:20.621205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:20.623077 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:20.623077 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:20.623077 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:09:20.623077 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:09:20.623077 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:09:20.623077 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:09:20.623077 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:09:20.623077 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:09:20.623077 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 19:09:20.623077 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:20.629482 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 390/794] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Jul 25 19:09:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 589 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:20.637636 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fca34-cbe3-4d05-a2cb-076eb5d28502 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:20.672629 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5407307-43db-4763-8d19-71879ba9d1cb None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:20.687582 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:20.770745 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d9c94-aa56-4ed2-9ee5-7ab5ab62449f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:20.772319 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d9c94-aa56-4ed2-9ee5-7ab5ab62449f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:20.772833 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d9c94-aa56-4ed2-9ee5-7ab5ab62449f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:20.773317 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d9c94-aa56-4ed2-9ee5-7ab5ab62449f None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:20.774942 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3d9c94-aa56-4ed2-9ee5-7ab5ab62449f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:20.775715 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3d9c94-aa56-4ed2-9ee5-7ab5ab62449f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:20.779736 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3d9c94-aa56-4ed2-9ee5-7ab5ab62449f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:20.824807 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 405/795] 10.222.0.18 () {56 vars in 1149 bytes} [Mon Jul 25 19:09:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 465 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:20.829551 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a72b0ee-5457-4841-8b8e-b4ffa938020a None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:20.829551 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a72b0ee-5457-4841-8b8e-b4ffa938020a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:20.829551 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a72b0ee-5457-4841-8b8e-b4ffa938020a None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:20.832771 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 406/796] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:20.834261 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a72b0ee-5457-4841-8b8e-b4ffa938020a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:20.896464 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fca34-cbe3-4d05-a2cb-076eb5d28502 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:20.972544 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3629a-67d0-4a06-bd8f-2cf603b12851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:20.974378 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3629a-67d0-4a06-bd8f-2cf603b12851 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:20.975407 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3629a-67d0-4a06-bd8f-2cf603b12851 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:20.976280 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3629a-67d0-4a06-bd8f-2cf603b12851 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:20.978467 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3629a-67d0-4a06-bd8f-2cf603b12851 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:20.980124 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3629a-67d0-4a06-bd8f-2cf603b12851 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:20.983259 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3629a-67d0-4a06-bd8f-2cf603b12851 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:21.020852 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 391/797] 10.222.0.18 () {56 vars in 1135 bytes} [Mon Jul 25 19:09:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:09:21.024167 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd51d23-2102-458c-9295-79e26563f7fa None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:21.025081 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd51d23-2102-458c-9295-79e26563f7fa None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:21.026040 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd51d23-2102-458c-9295-79e26563f7fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:21.061967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3bd51d23-2102-458c-9295-79e26563f7fa None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:21.064908 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3bd51d23-2102-458c-9295-79e26563f7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:21.080533 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a72b0ee-5457-4841-8b8e-b4ffa938020a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:21.139113 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:21.160157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73bb2e3-3125-4c28-92ef-d10662a0b13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:21.162389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bb2e3-3125-4c28-92ef-d10662a0b13f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:21.163785 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bb2e3-3125-4c28-92ef-d10662a0b13f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:21.164544 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bb2e3-3125-4c28-92ef-d10662a0b13f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:21.187636 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73bb2e3-3125-4c28-92ef-d10662a0b13f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:21.189731 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73bb2e3-3125-4c28-92ef-d10662a0b13f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:21.200026 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 392/798] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:09:21.202172 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd51d23-2102-458c-9295-79e26563f7fa None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:21.208072 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:21.212568 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-c73bb2e3-3125-4c28-92ef-d10662a0b13f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:21.212568 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:21.212568 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:21.212568 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:21.212568 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:21.212568 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:21.212568 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:21.212568 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:21.212568 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:21.212568 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:21.212568 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:21.212568 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:21.212568 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:09:21.212568 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:09:21.212568 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:09:21.215051 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:09:21.215051 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:21.215051 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:21.215051 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:21.215051 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:21.215051 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:21.215051 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:21.215051 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:21.215051 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:21.215051 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:21.215051 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:21.215051 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:21.215051 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:21.215051 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:21.215051 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:21.215051 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:21.215051 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:21.217188 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:09:21.217188 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:09:21.217188 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:21.217188 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:21.224894 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 407/799] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:09:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:21.224894 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b9b6f-5672-45e9-b9fb-6bc309d24d79 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:21.224894 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b9b6f-5672-45e9-b9fb-6bc309d24d79 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:21.224894 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b9b6f-5672-45e9-b9fb-6bc309d24d79 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:21.224894 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 408/800] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:21.226322 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5b9b6f-5672-45e9-b9fb-6bc309d24d79 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:21.366574 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd51d23-2102-458c-9295-79e26563f7fa None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:21.388123 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5b9b6f-5672-45e9-b9fb-6bc309d24d79 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:21.418136 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dae36f-fcf6-489d-8b86-462dbc804963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:21.419835 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dae36f-fcf6-489d-8b86-462dbc804963 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:21.420348 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dae36f-fcf6-489d-8b86-462dbc804963 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:21.420848 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dae36f-fcf6-489d-8b86-462dbc804963 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:21.436382 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010dfd21-5fc1-4fde-a5eb-01dc6d1ba4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:21.437076 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010dfd21-5fc1-4fde-a5eb-01dc6d1ba4a8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:21.437364 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010dfd21-5fc1-4fde-a5eb-01dc6d1ba4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:21.437656 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010dfd21-5fc1-4fde-a5eb-01dc6d1ba4a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:21.438401 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010dfd21-5fc1-4fde-a5eb-01dc6d1ba4a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:21.438716 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010dfd21-5fc1-4fde-a5eb-01dc6d1ba4a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:21.440284 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010dfd21-5fc1-4fde-a5eb-01dc6d1ba4a8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:21.441892 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dae36f-fcf6-489d-8b86-462dbc804963 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:21.444821 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dae36f-fcf6-489d-8b86-462dbc804963 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:21.460766 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 409/801] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:09:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:09:21.460766 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8e615-c210-49da-875c-e0d1747e05eb None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:21.460766 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8e615-c210-49da-875c-e0d1747e05eb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:21.460766 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8e615-c210-49da-875c-e0d1747e05eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:21.481366 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ecf8e615-c210-49da-875c-e0d1747e05eb None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:21.482543 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ecf8e615-c210-49da-875c-e0d1747e05eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:21.487002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-30dae36f-fcf6-489d-8b86-462dbc804963 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:21.487002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:21.487002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:21.487002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:21.487002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:21.487002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:21.487002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:21.487002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:21.487002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:21.487002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:21.487002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:21.487002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:21.487002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:09:21.487002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:09:21.487002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:09:21.488129 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:09:21.488129 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:21.488129 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:21.488129 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:21.488129 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:21.488129 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:21.488129 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:21.488129 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:21.488129 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:21.488129 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:21.488129 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:21.488129 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:21.488129 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:21.488129 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:21.488129 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:21.488129 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:21.488129 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:21.489170 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:09:21.489170 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:09:21.489170 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:21.489170 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:21.491913 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 393/802] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:09:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:21.492654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dae36f-fcf6-489d-8b86-462dbc804963 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:21.535369 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:21.582011 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 410/803] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:09:21.583119 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf8e615-c210-49da-875c-e0d1747e05eb None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:21.592436 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:21.655224 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dae36f-fcf6-489d-8b86-462dbc804963 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:21.701710 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b654d-2ac6-4396-88cc-bd4d1994d114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:21.702660 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b654d-2ac6-4396-88cc-bd4d1994d114 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:21.702953 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b654d-2ac6-4396-88cc-bd4d1994d114 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:21.703264 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b654d-2ac6-4396-88cc-bd4d1994d114 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:21.704000 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b654d-2ac6-4396-88cc-bd4d1994d114 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:21.705422 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b654d-2ac6-4396-88cc-bd4d1994d114 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:21.719929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-ed8b654d-2ac6-4396-88cc-bd4d1994d114 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 19:09:21.719929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:21.719929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 19:09:21.719929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 19:09:21.719929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 19:09:21.719929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 19:09:21.719929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 19:09:21.719929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:21.719929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 19:09:21.719929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:21.719929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 19:09:21.719929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:21.719929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:21.719929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:21.719929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:21.719929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:21.721187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:21.721187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:21.721187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:21.721187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:21.721187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:21.721187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:21.721187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:21.721187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 19:09:21.721187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 19:09:21.721187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 19:09:21.721187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 19:09:21.721187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:21.721187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:21.721187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:21.721187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:21.721187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:21.729434 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:21.729434 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:21.729434 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:21.729434 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:21.729434 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:21.729434 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:21.729434 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:21.729434 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:21.729434 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:21.729434 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:21.729434 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:21.729434 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 19:09:21.729434 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:21.729434 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 19:09:21.729434 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:21.730824 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf8e615-c210-49da-875c-e0d1747e05eb None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:21.730975 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 394/804] 10.222.0.18 () {56 vars in 1129 bytes} [Mon Jul 25 19:09:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:21.730975 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b654d-2ac6-4396-88cc-bd4d1994d114 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:21.788861 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bfedf3-4eb4-49eb-a46d-64771a50137c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:21.789696 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bfedf3-4eb4-49eb-a46d-64771a50137c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:21.789986 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bfedf3-4eb4-49eb-a46d-64771a50137c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:21.790271 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bfedf3-4eb4-49eb-a46d-64771a50137c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:21.798653 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bfedf3-4eb4-49eb-a46d-64771a50137c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:21.800193 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bfedf3-4eb4-49eb-a46d-64771a50137c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:21.815366 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-75bfedf3-4eb4-49eb-a46d-64771a50137c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:09:21.815366 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:21.815366 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:21.815366 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:21.815366 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:21.815366 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:21.815366 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:21.815366 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:21.815366 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:21.815366 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:21.815366 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:21.815366 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:21.815366 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:09:21.815366 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:09:21.815366 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:09:21.816675 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:09:21.816675 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:21.816675 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:21.816675 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:21.816675 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:21.816675 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:21.816675 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:21.816675 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:21.816675 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:21.816675 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:21.816675 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:21.816675 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:21.816675 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:21.816675 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:21.816675 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:21.816675 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:21.816675 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:21.817713 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:09:21.817713 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:09:21.817713 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:09:21.817713 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:09:21.817713 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:09:21.817713 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:09:21.817713 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:09:21.817713 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:21.817713 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 411/805] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:09:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:21.818298 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bfedf3-4eb4-49eb-a46d-64771a50137c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:21.872113 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:21.886758 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b654d-2ac6-4396-88cc-bd4d1994d114 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:21.915230 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:21.937424 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6b59d-8ed0-4b79-9813-859d39e9c690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:21.938113 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6b59d-8ed0-4b79-9813-859d39e9c690 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:21.938402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6b59d-8ed0-4b79-9813-859d39e9c690 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:21.938686 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6b59d-8ed0-4b79-9813-859d39e9c690 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:21.940301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d6b59d-8ed0-4b79-9813-859d39e9c690 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:21.940301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d6b59d-8ed0-4b79-9813-859d39e9c690 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:21.941400 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d6b59d-8ed0-4b79-9813-859d39e9c690 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:21.963247 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 395/806] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:09:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:21.964725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6b59d-8ed0-4b79-9813-859d39e9c690 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:22.008169 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bfedf3-4eb4-49eb-a46d-64771a50137c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:22.074793 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f8a2bd-75d7-431b-a4e8-bbe8ee7380e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:22.075834 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f8a2bd-75d7-431b-a4e8-bbe8ee7380e0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:22.076245 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f8a2bd-75d7-431b-a4e8-bbe8ee7380e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:22.076706 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f8a2bd-75d7-431b-a4e8-bbe8ee7380e0 None admin] PATH_INFO: `/v3/services/network` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:22.080895 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f8a2bd-75d7-431b-a4e8-bbe8ee7380e0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:22.083584 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f8a2bd-75d7-431b-a4e8-bbe8ee7380e0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:22.100528 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-f3f8a2bd-75d7-431b-a4e8-bbe8ee7380e0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 19:09:22.100528 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:22.100528 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 19:09:22.100528 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 19:09:22.100528 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 19:09:22.100528 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 19:09:22.100528 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 19:09:22.100528 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:22.100528 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 19:09:22.100528 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:22.100528 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 19:09:22.100528 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:22.100528 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:22.100528 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:22.100528 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:22.100528 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:22.100528 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:22.108329 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:22.108329 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:22.108329 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:22.108329 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:22.108329 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:22.108329 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:22.108329 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 19:09:22.108329 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 19:09:22.108329 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 19:09:22.108329 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 19:09:22.108329 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:22.108329 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:22.108329 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:22.108329 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:22.108329 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:22.108329 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:22.109532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:22.109532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:22.109532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:22.109532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:22.109532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:22.109532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:22.109532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:22.109532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:22.109532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:22.109532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:22.109532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 19:09:22.109532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:22.109532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 19:09:22.109532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:22.109532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 412/807] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:09:21 2022] GET /identity/v3/services/network => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:22.110422 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f8a2bd-75d7-431b-a4e8-bbe8ee7380e0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:22.140265 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6b59d-8ed0-4b79-9813-859d39e9c690 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:22.186146 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea616b3-20b3-44a8-acd6-f9445b49712e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:22.186950 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea616b3-20b3-44a8-acd6-f9445b49712e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:22.187150 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea616b3-20b3-44a8-acd6-f9445b49712e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:22.187431 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea616b3-20b3-44a8-acd6-f9445b49712e None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:22.188198 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea616b3-20b3-44a8-acd6-f9445b49712e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:22.188542 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea616b3-20b3-44a8-acd6-f9445b49712e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:22.189977 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea616b3-20b3-44a8-acd6-f9445b49712e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:22.195548 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 396/808] 10.222.0.18 () {56 vars in 1124 bytes} [Mon Jul 25 19:09:21 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:22.197109 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d3e79a-d514-4150-a1fa-08229664229e None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:22.197280 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d3e79a-d514-4150-a1fa-08229664229e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:22.197557 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d3e79a-d514-4150-a1fa-08229664229e None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:22.201968 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 397/809] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:22.201968 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d3e79a-d514-4150-a1fa-08229664229e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:22.255372 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:22.265430 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:22.270354 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f8a2bd-75d7-431b-a4e8-bbe8ee7380e0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:22.329248 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd052dd3-e5cc-42d3-a2ef-b4e713db7bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:22.330027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd052dd3-e5cc-42d3-a2ef-b4e713db7bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:22.330477 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd052dd3-e5cc-42d3-a2ef-b4e713db7bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:22.330854 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd052dd3-e5cc-42d3-a2ef-b4e713db7bf9 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:22.331911 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd052dd3-e5cc-42d3-a2ef-b4e713db7bf9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:22.332360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd052dd3-e5cc-42d3-a2ef-b4e713db7bf9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:22.334044 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd052dd3-e5cc-42d3-a2ef-b4e713db7bf9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:22.353292 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 413/810] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Jul 25 19:09:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:22.354206 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd052dd3-e5cc-42d3-a2ef-b4e713db7bf9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:22.358761 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d3e79a-d514-4150-a1fa-08229664229e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:22.412022 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce5623-6662-4276-b2ac-d3829ef64e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:22.413247 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce5623-6662-4276-b2ac-d3829ef64e8a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:22.413712 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce5623-6662-4276-b2ac-d3829ef64e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:22.414163 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce5623-6662-4276-b2ac-d3829ef64e8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:22.426837 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adce5623-6662-4276-b2ac-d3829ef64e8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:22.429075 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adce5623-6662-4276-b2ac-d3829ef64e8a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:22.453978 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-adce5623-6662-4276-b2ac-d3829ef64e8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:09:22.453978 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:22.453978 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:22.453978 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:22.453978 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:22.453978 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:22.453978 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:22.453978 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:22.453978 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:22.453978 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:22.453978 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:22.453978 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:22.453978 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:09:22.453978 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:09:22.453978 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:09:22.455025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:09:22.455025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:22.455025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:22.455025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:22.455025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:22.455025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:22.455025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:22.455025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:22.455025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:22.455025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:22.455025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:22.455025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:22.455025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:22.455025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:22.455025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:22.455025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:22.455025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:22.456451 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:09:22.456451 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:09:22.456451 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:09:22.456451 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:09:22.456451 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:09:22.456451 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:09:22.456451 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:09:22.456451 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:22.456451 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 398/811] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:09:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:22.457035 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce5623-6662-4276-b2ac-d3829ef64e8a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:22.517670 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd052dd3-e5cc-42d3-a2ef-b4e713db7bf9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:22.567051 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80bf80a-e445-4d0c-a721-f49bd7bfb4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:22.568771 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bf80a-e445-4d0c-a721-f49bd7bfb4b4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:22.569072 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bf80a-e445-4d0c-a721-f49bd7bfb4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:22.569396 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bf80a-e445-4d0c-a721-f49bd7bfb4b4 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:22.570173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80bf80a-e445-4d0c-a721-f49bd7bfb4b4 None admin] RBAC: Adding query filter params (name=network) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:22.570487 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80bf80a-e445-4d0c-a721-f49bd7bfb4b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:22.571940 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80bf80a-e445-4d0c-a721-f49bd7bfb4b4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:22.583034 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 414/812] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Jul 25 19:09:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:22.583034 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed734cc-7fe0-465e-b8c9-67bb76b2418e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:22.583034 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed734cc-7fe0-465e-b8c9-67bb76b2418e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:22.583034 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed734cc-7fe0-465e-b8c9-67bb76b2418e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:22.603726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9ed734cc-7fe0-465e-b8c9-67bb76b2418e None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:22.605126 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9ed734cc-7fe0-465e-b8c9-67bb76b2418e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:22.621369 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:22.625495 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce5623-6662-4276-b2ac-d3829ef64e8a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:22.629988 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:22.681523 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a943e82-3ca2-4192-8590-cfa89d1282f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:22.682635 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a943e82-3ca2-4192-8590-cfa89d1282f1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:22.683059 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a943e82-3ca2-4192-8590-cfa89d1282f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:22.683592 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a943e82-3ca2-4192-8590-cfa89d1282f1 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:22.684947 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a943e82-3ca2-4192-8590-cfa89d1282f1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:22.685517 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a943e82-3ca2-4192-8590-cfa89d1282f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:22.688113 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a943e82-3ca2-4192-8590-cfa89d1282f1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:22.692854 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 415/813] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:09:22.693724 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed734cc-7fe0-465e-b8c9-67bb76b2418e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:22.717363 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 399/814] 10.222.0.18 () {56 vars in 1124 bytes} [Mon Jul 25 19:09:22 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:22.717363 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a943e82-3ca2-4192-8590-cfa89d1282f1 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:22.820706 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed734cc-7fe0-465e-b8c9-67bb76b2418e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:22.864745 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85c5b8-945f-4a3a-b601-e8302f093f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:22.865860 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85c5b8-945f-4a3a-b601-e8302f093f68 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:22.866286 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85c5b8-945f-4a3a-b601-e8302f093f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:22.866658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85c5b8-945f-4a3a-b601-e8302f093f68 None admin] PATH_INFO: `/v3/projects/191069caccd04233b012585e29cfa18b/users/1c0772205ae24713b8901666a87018b2/roles/23718d83b0864b16b0071e8ffe0b5861` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:22.886907 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a943e82-3ca2-4192-8590-cfa89d1282f1 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:22.891975 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e85c5b8-945f-4a3a-b601-e8302f093f68 None admin] RBAC: Authorizing `identity:create_grant(project_id=191069caccd04233b012585e29cfa18b, user_id=1c0772205ae24713b8901666a87018b2, role_id=23718d83b0864b16b0071e8ffe0b5861)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:22.893689 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e85c5b8-945f-4a3a-b601-e8302f093f68 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:22.905846 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:22.913011 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 416/815] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Jul 25 19:09:22 2022] PUT /identity/v3/projects/191069caccd04233b012585e29cfa18b/users/1c0772205ae24713b8901666a87018b2/roles/23718d83b0864b16b0071e8ffe0b5861 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:09:22.923660 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85c5b8-945f-4a3a-b601-e8302f093f68 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:22.946641 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c6d34-5e8e-481b-83c7-31992160498c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:22.947270 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c6d34-5e8e-481b-83c7-31992160498c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:22.947669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c6d34-5e8e-481b-83c7-31992160498c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:22.947996 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c6d34-5e8e-481b-83c7-31992160498c None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:22.949505 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c6d34-5e8e-481b-83c7-31992160498c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:22.949505 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c6d34-5e8e-481b-83c7-31992160498c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:22.951160 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c6d34-5e8e-481b-83c7-31992160498c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:22.975915 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 400/816] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Jul 25 19:09:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:22.975915 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c6d34-5e8e-481b-83c7-31992160498c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:23.022487 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:23.118737 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85c5b8-945f-4a3a-b601-e8302f093f68 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:23.149957 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c6d34-5e8e-481b-83c7-31992160498c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:23.191221 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de44c0-3633-4f6c-9404-273b3f103781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:23.192367 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de44c0-3633-4f6c-9404-273b3f103781 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:23.192822 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de44c0-3633-4f6c-9404-273b3f103781 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:23.193340 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de44c0-3633-4f6c-9404-273b3f103781 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:23.194452 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3de44c0-3633-4f6c-9404-273b3f103781 None admin] RBAC: Adding query filter params (type=network) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:23.194966 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3de44c0-3633-4f6c-9404-273b3f103781 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:23.197530 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3de44c0-3633-4f6c-9404-273b3f103781 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:23.216622 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 417/817] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Jul 25 19:09:22 2022] GET /identity/v3/services?type=network => generated 350 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:23.216622 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de44c0-3633-4f6c-9404-273b3f103781 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:23.222883 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ee346-449d-4f23-84c5-fb69b76565df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:23.223942 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ee346-449d-4f23-84c5-fb69b76565df None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:23.224241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ee346-449d-4f23-84c5-fb69b76565df None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:23.224705 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ee346-449d-4f23-84c5-fb69b76565df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:23.240137 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ee346-449d-4f23-84c5-fb69b76565df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:09:23.248714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-396ee346-449d-4f23-84c5-fb69b76565df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:09:23.248714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:23.248714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:23.248714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:23.248714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:23.248714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:23.248714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:23.248714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:23.248714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:23.248714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:23.248714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:23.248714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:23.248714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:09:23.248714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:09:23.248714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:09:23.248714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:09:23.249782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:09:23.249782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:09:23.249782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:09:23.249782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:09:23.249782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:23.249782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:23.249782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:23.249782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:23.249782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:23.249782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:23.249782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:23.249782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:23.249782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:23.249782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:23.249782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:23.249782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:23.250757 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:23.250757 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:23.250757 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:23.250757 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:23.250757 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:09:23.250757 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:09:23.250757 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:09:23.250757 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:09:23.250757 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:09:23.250757 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:09:23.250757 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:09:23.250757 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:23.250757 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 401/818] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Jul 25 19:09:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:23.251560 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ee346-449d-4f23-84c5-fb69b76565df None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:23.329867 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:23.402057 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de44c0-3633-4f6c-9404-273b3f103781 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:23.406059 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:23.431371 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ee346-449d-4f23-84c5-fb69b76565df None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:23.475138 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15d07b-ad5b-44df-bf6c-d3782e8d2525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:23.475971 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15d07b-ad5b-44df-bf6c-d3782e8d2525 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:23.476453 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15d07b-ad5b-44df-bf6c-d3782e8d2525 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:23.476922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15d07b-ad5b-44df-bf6c-d3782e8d2525 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:23.478025 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15d07b-ad5b-44df-bf6c-d3782e8d2525 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:23.480630 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15d07b-ad5b-44df-bf6c-d3782e8d2525 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:23.508250 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562adc01-96a6-4c53-bd60-d4755d93fc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:23.509618 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562adc01-96a6-4c53-bd60-d4755d93fc6a None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:23.510935 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562adc01-96a6-4c53-bd60-d4755d93fc6a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:23.511504 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562adc01-96a6-4c53-bd60-d4755d93fc6a None admin] PATH_INFO: `/v3/projects/191069caccd04233b012585e29cfa18b/users/feafcf1b5a9844fe9738823fddd07e0e/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:23.514696 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562adc01-96a6-4c53-bd60-d4755d93fc6a None admin] RBAC: Authorizing `identity:create_grant(project_id=191069caccd04233b012585e29cfa18b, user_id=feafcf1b5a9844fe9738823fddd07e0e, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:23.516858 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562adc01-96a6-4c53-bd60-d4755d93fc6a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:23.527928 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 418/819] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:09:23 2022] POST /identity/v3/endpoints => generated 346 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:09:23.527928 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15d07b-ad5b-44df-bf6c-d3782e8d2525 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:23.551300 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 402/820] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Jul 25 19:09:23 2022] PUT /identity/v3/projects/191069caccd04233b012585e29cfa18b/users/feafcf1b5a9844fe9738823fddd07e0e/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:09:23.553849 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562adc01-96a6-4c53-bd60-d4755d93fc6a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:23.801257 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15d07b-ad5b-44df-bf6c-d3782e8d2525 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:23.806989 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562adc01-96a6-4c53-bd60-d4755d93fc6a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:23.847301 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:23.886345 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a3676e-23ad-4cbe-bb9d-47eb0720f268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:23.887056 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3676e-23ad-4cbe-bb9d-47eb0720f268 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:23.887469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3676e-23ad-4cbe-bb9d-47eb0720f268 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:23.887720 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3676e-23ad-4cbe-bb9d-47eb0720f268 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:23.888442 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a3676e-23ad-4cbe-bb9d-47eb0720f268 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:23.889228 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a3676e-23ad-4cbe-bb9d-47eb0720f268 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:23.890230 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a3676e-23ad-4cbe-bb9d-47eb0720f268 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:23.900866 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 403/821] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:09:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:23.907179 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a3676e-23ad-4cbe-bb9d-47eb0720f268 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:23.907397 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7d9d9-4e74-44bf-ba75-6690675fa41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:23.908877 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7d9d9-4e74-44bf-ba75-6690675fa41e None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:23.909368 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7d9d9-4e74-44bf-ba75-6690675fa41e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:23.909868 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7d9d9-4e74-44bf-ba75-6690675fa41e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:23.911349 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e7d9d9-4e74-44bf-ba75-6690675fa41e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:23.914102 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e7d9d9-4e74-44bf-ba75-6690675fa41e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:23.925123 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:23.953568 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 419/822] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:09:23 2022] POST /identity/v3/endpoints => generated 327 bytes in 431 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:09:24.088471 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a3676e-23ad-4cbe-bb9d-47eb0720f268 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:24.176516 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14275226-76a3-4d1c-9045-54bdab2bd23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:24.177467 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14275226-76a3-4d1c-9045-54bdab2bd23f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:24.177753 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14275226-76a3-4d1c-9045-54bdab2bd23f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:24.178128 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14275226-76a3-4d1c-9045-54bdab2bd23f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:24.198619 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14275226-76a3-4d1c-9045-54bdab2bd23f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:24.200399 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14275226-76a3-4d1c-9045-54bdab2bd23f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:24.236214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-14275226-76a3-4d1c-9045-54bdab2bd23f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:24.236214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:24.236214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:24.236214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:24.236214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:24.236214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:24.236214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:24.236214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:24.236214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:24.236214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:24.236214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:24.236214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:24.236214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:09:24.236214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:09:24.236214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:09:24.237205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:09:24.237205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:24.237205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:24.237205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:24.237205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:24.237205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:24.237205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:24.237205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:24.237205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:24.237205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:24.237205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:24.237205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:24.237205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:24.237205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:24.237205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:24.237205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:24.237205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:24.238134 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:09:24.238134 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:09:24.238134 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:24.238134 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:24.238134 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 404/823] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:09:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:09:24.245567 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7d9d9-4e74-44bf-ba75-6690675fa41e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:24.444275 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7d9d9-4e74-44bf-ba75-6690675fa41e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:24.501255 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ead28a-3f38-461a-bc88-02395bacd246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:24.502029 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ead28a-3f38-461a-bc88-02395bacd246 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:24.502314 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ead28a-3f38-461a-bc88-02395bacd246 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:24.502621 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ead28a-3f38-461a-bc88-02395bacd246 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:24.503395 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ead28a-3f38-461a-bc88-02395bacd246 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:24.503754 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ead28a-3f38-461a-bc88-02395bacd246 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:24.505230 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ead28a-3f38-461a-bc88-02395bacd246 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:24.529518 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 420/824] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:09:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:24.538583 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14275226-76a3-4d1c-9045-54bdab2bd23f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:24.687990 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:24.771861 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14275226-76a3-4d1c-9045-54bdab2bd23f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:24.834870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee2e95-03e6-4529-a86b-4e485ab97c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:24.836706 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee2e95-03e6-4529-a86b-4e485ab97c25 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:24.837468 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee2e95-03e6-4529-a86b-4e485ab97c25 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:24.838240 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee2e95-03e6-4529-a86b-4e485ab97c25 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:24.854411 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee2e95-03e6-4529-a86b-4e485ab97c25 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:24.857092 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee2e95-03e6-4529-a86b-4e485ab97c25 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:24.873936 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-3dee2e95-03e6-4529-a86b-4e485ab97c25 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 19:09:24.873936 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:24.873936 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:24.873936 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:24.873936 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:24.873936 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:24.873936 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:24.873936 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:24.873936 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:24.873936 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:24.873936 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:24.873936 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:24.873936 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:09:24.873936 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:09:24.873936 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:09:24.873936 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:09:24.875199 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:24.875199 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:24.875199 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:09:24.875199 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:09:24.875199 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:09:24.875199 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:09:24.875199 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:24.875199 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:24.875199 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:24.875199 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:24.875199 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:24.875199 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:24.875199 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:24.875199 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:24.875199 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:24.875199 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:24.875199 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:24.875199 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:24.876621 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:24.876621 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:24.876621 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:09:24.876621 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:09:24.876621 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:09:24.876621 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:09:24.876621 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:09:24.876621 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:09:24.876621 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 19:09:24.876621 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:24.879217 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 405/825] 10.222.0.18 () {56 vars in 1093 bytes} [Mon Jul 25 19:09:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:24.889189 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ead28a-3f38-461a-bc88-02395bacd246 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:24.956101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:25.117697 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ead28a-3f38-461a-bc88-02395bacd246 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:25.189270 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f070ba-ad20-46cb-88f0-599d35385099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:25.189967 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f070ba-ad20-46cb-88f0-599d35385099 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:25.190263 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f070ba-ad20-46cb-88f0-599d35385099 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:25.190518 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f070ba-ad20-46cb-88f0-599d35385099 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:25.191427 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f070ba-ad20-46cb-88f0-599d35385099 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:25.191892 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f070ba-ad20-46cb-88f0-599d35385099 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:25.194462 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f070ba-ad20-46cb-88f0-599d35385099 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:25.222085 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 421/826] 10.222.0.18 () {56 vars in 1133 bytes} [Mon Jul 25 19:09:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:25.232550 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee2e95-03e6-4529-a86b-4e485ab97c25 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:25.481019 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee2e95-03e6-4529-a86b-4e485ab97c25 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:25.527123 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:25.564641 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84522933-378f-48a1-bc56-db8de141711a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:25.565867 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84522933-378f-48a1-bc56-db8de141711a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:25.566369 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84522933-378f-48a1-bc56-db8de141711a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:25.566870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84522933-378f-48a1-bc56-db8de141711a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:25.610058 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c8e19-f697-4cd8-b698-67feb9b98c5f None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:25.611365 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c8e19-f697-4cd8-b698-67feb9b98c5f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:25.612007 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c8e19-f697-4cd8-b698-67feb9b98c5f None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:25.614374 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 422/827] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:09:25.621606 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea60b14-3862-4199-8134-cf0fd657ebab None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:25.622298 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea60b14-3862-4199-8134-cf0fd657ebab None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:25.622934 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea60b14-3862-4199-8134-cf0fd657ebab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:25.646031 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-aea60b14-3862-4199-8134-cf0fd657ebab None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:25.647958 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-aea60b14-3862-4199-8134-cf0fd657ebab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:25.733316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84522933-378f-48a1-bc56-db8de141711a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:25.735785 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84522933-378f-48a1-bc56-db8de141711a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:25.758649 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 423/828] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:25 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 19:09:25.779257 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea60b14-3862-4199-8134-cf0fd657ebab None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:25.806256 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:25.938254 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 406/829] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:09:25 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 706 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 19:09:25.944949 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84522933-378f-48a1-bc56-db8de141711a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:25.992641 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:26.012687 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea60b14-3862-4199-8134-cf0fd657ebab None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:26.068877 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84273e3f-b1b0-4b02-82ec-826b240e91bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:26.070484 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84273e3f-b1b0-4b02-82ec-826b240e91bd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:26.071015 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84273e3f-b1b0-4b02-82ec-826b240e91bd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:26.072240 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84273e3f-b1b0-4b02-82ec-826b240e91bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:26.096175 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84273e3f-b1b0-4b02-82ec-826b240e91bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:09:26.113231 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-84273e3f-b1b0-4b02-82ec-826b240e91bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:09:26.113231 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:26.113231 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:26.113231 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:26.113231 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:26.113231 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:26.113231 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:26.113231 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:26.113231 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:26.113231 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:26.113231 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:26.113231 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:26.113231 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:09:26.113231 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:09:26.113231 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:09:26.113231 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:09:26.115327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:09:26.115327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:09:26.115327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:09:26.115327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:09:26.115327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:26.115327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:26.115327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:26.115327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:26.115327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:26.115327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:26.115327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:26.115327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:26.115327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:26.115327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:26.115327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:26.115327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:26.117310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:26.117310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:26.117310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:26.117310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:26.117310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:09:26.117310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:09:26.117310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:09:26.117310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:09:26.117310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:09:26.117310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:09:26.117310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:09:26.117310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:26.123496 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 424/830] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Jul 25 19:09:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:09:26.129955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84273e3f-b1b0-4b02-82ec-826b240e91bd None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:26.140701 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84522933-378f-48a1-bc56-db8de141711a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:26.218068 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ac6e3-6a80-411d-8d34-dfd7057db50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:26.220152 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ac6e3-6a80-411d-8d34-dfd7057db50e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:26.221084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ac6e3-6a80-411d-8d34-dfd7057db50e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:26.221879 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ac6e3-6a80-411d-8d34-dfd7057db50e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:26.246168 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ac6e3-6a80-411d-8d34-dfd7057db50e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:26.249610 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ac6e3-6a80-411d-8d34-dfd7057db50e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:26.253181 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:26.292449 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84273e3f-b1b0-4b02-82ec-826b240e91bd None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:26.292946 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-7a9ac6e3-6a80-411d-8d34-dfd7057db50e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:26.292946 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:26.292946 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:26.292946 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:26.292946 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:26.292946 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:26.292946 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:26.292946 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:26.292946 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:26.292946 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:26.292946 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:26.292946 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:26.292946 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:09:26.292946 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:09:26.292946 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:09:26.294627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:09:26.294627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:26.294627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:26.294627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:26.294627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:26.294627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:26.294627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:26.294627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:26.294627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:26.294627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:26.294627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:26.294627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:26.294627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:26.294627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:26.294627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:26.294627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:26.294627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:26.296475 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:09:26.296475 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:09:26.296475 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:26.296475 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:26.300642 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 407/831] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:09:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:09:26.308315 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ac6e3-6a80-411d-8d34-dfd7057db50e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:26.354713 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c1866-fdcd-4fba-85c9-05b8652c182a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:26.356650 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c1866-fdcd-4fba-85c9-05b8652c182a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:26.357554 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c1866-fdcd-4fba-85c9-05b8652c182a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:26.358383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c1866-fdcd-4fba-85c9-05b8652c182a None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:26.360310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c1866-fdcd-4fba-85c9-05b8652c182a None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:26.361240 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c1866-fdcd-4fba-85c9-05b8652c182a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:26.364394 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c1866-fdcd-4fba-85c9-05b8652c182a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:26.383751 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 425/832] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Jul 25 19:09:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:09:26.390881 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c1866-fdcd-4fba-85c9-05b8652c182a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:26.420206 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:26.461103 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ac6e3-6a80-411d-8d34-dfd7057db50e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:26.506994 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9985d515-60da-4d6a-bf53-dd5de4f9cdfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:26.507286 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9985d515-60da-4d6a-bf53-dd5de4f9cdfc None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:26.507431 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9985d515-60da-4d6a-bf53-dd5de4f9cdfc None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:26.507817 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9985d515-60da-4d6a-bf53-dd5de4f9cdfc None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:26.508580 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9985d515-60da-4d6a-bf53-dd5de4f9cdfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:26.508896 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9985d515-60da-4d6a-bf53-dd5de4f9cdfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:26.513592 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9985d515-60da-4d6a-bf53-dd5de4f9cdfc None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:26.531307 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 408/833] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:09:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:26.532423 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9985d515-60da-4d6a-bf53-dd5de4f9cdfc None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:26.610626 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c1866-fdcd-4fba-85c9-05b8652c182a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:26.660839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32824fcd-17c1-4f0f-8fab-3b2d48792d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:26.660839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32824fcd-17c1-4f0f-8fab-3b2d48792d3e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:26.660839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32824fcd-17c1-4f0f-8fab-3b2d48792d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:26.660839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32824fcd-17c1-4f0f-8fab-3b2d48792d3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:26.675949 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:26.679204 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32824fcd-17c1-4f0f-8fab-3b2d48792d3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:26.681698 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32824fcd-17c1-4f0f-8fab-3b2d48792d3e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:26.706387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-32824fcd-17c1-4f0f-8fab-3b2d48792d3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:26.706387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:26.706387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:26.706387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:26.706387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:26.706387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:26.706387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:26.706387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:26.706387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:26.706387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:26.706387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:26.706387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:26.706387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:09:26.706387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:09:26.706387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:09:26.708677 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:09:26.708677 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:26.708677 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:26.708677 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:26.708677 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:26.708677 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:26.708677 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:26.708677 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:26.708677 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:26.708677 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:26.708677 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:26.708677 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:26.708677 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:26.708677 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:26.708677 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:26.708677 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:26.708677 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:26.713359 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:09:26.713359 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:09:26.713359 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:26.713359 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:26.713359 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 426/834] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:09:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:26.713359 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d4fa8-fd64-41f7-85de-cb21e9b690a8 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:26.713359 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d4fa8-fd64-41f7-85de-cb21e9b690a8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:26.713359 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d4fa8-fd64-41f7-85de-cb21e9b690a8 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:26.722089 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 427/835] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:26 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:26.723225 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d4fa8-fd64-41f7-85de-cb21e9b690a8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:26.725534 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9985d515-60da-4d6a-bf53-dd5de4f9cdfc None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:26.777765 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:26.801235 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec3a20-6a86-445a-b3ed-afef25136bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:26.802935 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec3a20-6a86-445a-b3ed-afef25136bbf None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:26.803399 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec3a20-6a86-445a-b3ed-afef25136bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:26.803926 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec3a20-6a86-445a-b3ed-afef25136bbf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:26.814622 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ec3a20-6a86-445a-b3ed-afef25136bbf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:26.816118 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ec3a20-6a86-445a-b3ed-afef25136bbf None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:26.833887 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-34ec3a20-6a86-445a-b3ed-afef25136bbf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:09:26.833887 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:26.833887 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:26.833887 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:26.833887 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:26.833887 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:26.833887 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:26.833887 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:26.833887 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:26.833887 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:26.833887 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:26.833887 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:26.833887 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:09:26.833887 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:09:26.833887 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:09:26.834876 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:09:26.834876 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:26.834876 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:26.834876 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:26.834876 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:26.834876 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:26.834876 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:26.834876 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:26.834876 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:26.834876 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:26.834876 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:26.834876 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:26.834876 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:26.834876 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:26.834876 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:26.834876 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:26.834876 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:26.836826 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:09:26.836826 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:09:26.836826 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:09:26.836826 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:09:26.836826 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:09:26.836826 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:09:26.836826 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:09:26.836826 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:26.836826 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 409/836] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:09:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:26.836826 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0015e-eeb6-40ba-8533-8d75e18191fb None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:26.837443 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0015e-eeb6-40ba-8533-8d75e18191fb None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:26.837443 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0015e-eeb6-40ba-8533-8d75e18191fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:26.870876 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-24b0015e-eeb6-40ba-8533-8d75e18191fb None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:26.873057 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-24b0015e-eeb6-40ba-8533-8d75e18191fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:26.908426 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d4fa8-fd64-41f7-85de-cb21e9b690a8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:26.961426 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79efbb5c-130e-4b3b-8d25-166cd19920d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:26.962108 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efbb5c-130e-4b3b-8d25-166cd19920d8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:26.962389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efbb5c-130e-4b3b-8d25-166cd19920d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:26.962664 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efbb5c-130e-4b3b-8d25-166cd19920d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:26.963496 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79efbb5c-130e-4b3b-8d25-166cd19920d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:26.964499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79efbb5c-130e-4b3b-8d25-166cd19920d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:26.965991 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79efbb5c-130e-4b3b-8d25-166cd19920d8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:26.987038 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 428/837] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:09:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:26.987038 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79efbb5c-130e-4b3b-8d25-166cd19920d8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:26.992866 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 410/838] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:26 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:09:26.993812 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0015e-eeb6-40ba-8533-8d75e18191fb None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:27.087389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:27.176810 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79efbb5c-130e-4b3b-8d25-166cd19920d8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:27.187992 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0015e-eeb6-40ba-8533-8d75e18191fb None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:27.215095 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:27.236586 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c89c25-8f23-49b3-9253-672c7cedd9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:27.236973 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad4a79-b368-4958-bf3d-25d4e8a97de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:27.237436 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c89c25-8f23-49b3-9253-672c7cedd9b7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:27.237721 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c89c25-8f23-49b3-9253-672c7cedd9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:27.238001 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c89c25-8f23-49b3-9253-672c7cedd9b7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:27.238128 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad4a79-b368-4958-bf3d-25d4e8a97de1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:27.238555 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad4a79-b368-4958-bf3d-25d4e8a97de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:27.239049 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad4a79-b368-4958-bf3d-25d4e8a97de1 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:27.240597 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dad4a79-b368-4958-bf3d-25d4e8a97de1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:27.241173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dad4a79-b368-4958-bf3d-25d4e8a97de1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:27.244017 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dad4a79-b368-4958-bf3d-25d4e8a97de1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:27.247039 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c89c25-8f23-49b3-9253-672c7cedd9b7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:27.248587 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c89c25-8f23-49b3-9253-672c7cedd9b7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:27.271228 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 429/839] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Jul 25 19:09:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:27.271228 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad4a79-b368-4958-bf3d-25d4e8a97de1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:27.271662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-97c89c25-8f23-49b3-9253-672c7cedd9b7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 19:09:27.271662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:27.271662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:27.271662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:27.271662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:27.271662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:27.271662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:27.271662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:27.271662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:27.271662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:27.271662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:27.271662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:27.271662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:09:27.271662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:09:27.271662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:09:27.271662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:09:27.274089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:27.274089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:27.274089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:09:27.274089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:09:27.274089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:09:27.274089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:09:27.274089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:27.274089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:27.274089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:27.274089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:27.274089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:27.274089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:27.274089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:27.274089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:27.274089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:27.274089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:27.274089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:27.274089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:27.276208 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:27.276208 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:27.276208 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:09:27.276208 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:09:27.276208 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:09:27.276208 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:09:27.276208 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:09:27.276208 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:09:27.276208 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 19:09:27.276208 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:27.276208 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 411/840] 10.222.0.18 () {56 vars in 1117 bytes} [Mon Jul 25 19:09:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:27.276208 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdd603-95ed-4f83-a0a3-6bd01f7767e8 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:27.276208 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdd603-95ed-4f83-a0a3-6bd01f7767e8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:27.278434 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdd603-95ed-4f83-a0a3-6bd01f7767e8 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:27.278434 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 412/841] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:27.278434 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fdd603-95ed-4f83-a0a3-6bd01f7767e8 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:27.446149 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fdd603-95ed-4f83-a0a3-6bd01f7767e8 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:27.464346 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad4a79-b368-4958-bf3d-25d4e8a97de1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:27.482687 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:27.502617 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a801aa-2743-4160-acaf-b7190d4c5e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:27.503832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a801aa-2743-4160-acaf-b7190d4c5e75 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:27.504334 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a801aa-2743-4160-acaf-b7190d4c5e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:27.504835 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a801aa-2743-4160-acaf-b7190d4c5e75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:27.506533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a801aa-2743-4160-acaf-b7190d4c5e75 None admin] RBAC: Adding query filter params (role.id=ed7fec18968241efa76a0ee769d96da9, scope.project.id=191069caccd04233b012585e29cfa18b, user.id=860e7d26ad364795bb25836aadd4ac7b) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:27.507057 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a801aa-2743-4160-acaf-b7190d4c5e75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:27.509845 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a801aa-2743-4160-acaf-b7190d4c5e75 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:27.517690 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8daeb3-f030-47a8-946a-5da54b0f92a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:27.519145 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8daeb3-f030-47a8-946a-5da54b0f92a0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:27.519145 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8daeb3-f030-47a8-946a-5da54b0f92a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:27.519950 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8daeb3-f030-47a8-946a-5da54b0f92a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:27.522041 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 413/842] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:09:27 2022] GET /identity/v3/role_assignments?user.id=860e7d26ad364795bb25836aadd4ac7b&scope.project.id=191069caccd04233b012585e29cfa18b&role.id=ed7fec18968241efa76a0ee769d96da9 => generated 608 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:27.524010 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a801aa-2743-4160-acaf-b7190d4c5e75 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:27.551299 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8daeb3-f030-47a8-946a-5da54b0f92a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:09:27.566742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-bb8daeb3-f030-47a8-946a-5da54b0f92a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:09:27.566742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:27.566742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:27.566742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:27.566742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:27.566742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:27.566742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:27.566742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:27.566742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:27.566742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:27.566742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:27.566742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:27.566742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:09:27.566742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:09:27.566742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:09:27.566742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:09:27.568700 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:09:27.568700 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:09:27.568700 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:09:27.568700 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:09:27.568700 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:27.568700 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:27.568700 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:27.568700 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:27.568700 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:27.568700 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:27.568700 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:27.568700 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:27.568700 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:27.568700 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:27.568700 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:27.568700 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:27.581422 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:27.581422 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:27.581422 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:27.581422 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:27.581422 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:09:27.581422 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:09:27.581422 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:09:27.581422 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:09:27.581422 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:09:27.581422 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:09:27.581422 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:09:27.581422 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:27.581422 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 430/843] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Jul 25 19:09:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:27.581422 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b990e-6887-4252-b50d-6503be695afb None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:27.583224 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b990e-6887-4252-b50d-6503be695afb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:27.583224 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b990e-6887-4252-b50d-6503be695afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:27.616219 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6a5b990e-6887-4252-b50d-6503be695afb None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:27.618480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6a5b990e-6887-4252-b50d-6503be695afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:27.636643 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:27.762939 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a801aa-2743-4160-acaf-b7190d4c5e75 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:27.767021 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 431/844] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:27 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:09:27.767616 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b990e-6887-4252-b50d-6503be695afb None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:27.828153 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a9299-21bf-4238-a5ae-a043d4cedd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:27.829065 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a9299-21bf-4238-a5ae-a043d4cedd03 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:27.829360 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a9299-21bf-4238-a5ae-a043d4cedd03 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:27.829647 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a9299-21bf-4238-a5ae-a043d4cedd03 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:27.830549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192a9299-21bf-4238-a5ae-a043d4cedd03 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:27.830951 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192a9299-21bf-4238-a5ae-a043d4cedd03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:27.832409 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192a9299-21bf-4238-a5ae-a043d4cedd03 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:27.863561 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 414/845] 10.222.0.18 () {56 vars in 1149 bytes} [Mon Jul 25 19:09:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:27.863561 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a9299-21bf-4238-a5ae-a043d4cedd03 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:27.987872 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b990e-6887-4252-b50d-6503be695afb None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:27.991416 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:28.053482 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a9299-21bf-4238-a5ae-a043d4cedd03 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:28.056473 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:28.063087 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db9e87-6182-49af-98ce-1a6061ed6ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:28.064682 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db9e87-6182-49af-98ce-1a6061ed6ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:28.065265 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db9e87-6182-49af-98ce-1a6061ed6ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:28.065751 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db9e87-6182-49af-98ce-1a6061ed6ad0 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:28.067096 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db9e87-6182-49af-98ce-1a6061ed6ad0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:28.068300 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db9e87-6182-49af-98ce-1a6061ed6ad0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:28.070255 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db9e87-6182-49af-98ce-1a6061ed6ad0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:28.085736 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 432/846] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:09:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:28.087743 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db9e87-6182-49af-98ce-1a6061ed6ad0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:28.120806 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1dc7f0-7df9-4693-8cb4-454498d2facb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:28.122210 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1dc7f0-7df9-4693-8cb4-454498d2facb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:28.122704 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1dc7f0-7df9-4693-8cb4-454498d2facb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:28.123186 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1dc7f0-7df9-4693-8cb4-454498d2facb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:28.124543 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1dc7f0-7df9-4693-8cb4-454498d2facb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:28.127436 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1dc7f0-7df9-4693-8cb4-454498d2facb None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:28.150301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-be1dc7f0-7df9-4693-8cb4-454498d2facb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 19:09:28.150301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:28.150301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 19:09:28.150301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 19:09:28.150301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 19:09:28.150301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 19:09:28.150301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 19:09:28.150301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:28.150301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 19:09:28.150301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:28.150301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 19:09:28.150301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:28.150301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:28.150301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:28.150301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:28.150301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:28.150301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:28.153211 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:28.153211 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:28.153211 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:28.153211 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:28.153211 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:28.153211 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:28.153211 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 19:09:28.153211 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 19:09:28.153211 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 19:09:28.153211 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 19:09:28.153211 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:28.153211 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:28.153211 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:28.153211 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:28.153211 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:28.153211 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:28.154958 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:28.154958 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:28.154958 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:28.154958 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:28.154958 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:28.154958 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:28.154958 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:28.154958 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:28.154958 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:28.154958 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:28.154958 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 19:09:28.154958 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:28.154958 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 19:09:28.154958 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:28.154958 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 415/847] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Jul 25 19:09:27 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:28.158527 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1dc7f0-7df9-4693-8cb4-454498d2facb None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:28.305231 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db9e87-6182-49af-98ce-1a6061ed6ad0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:28.341620 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1dc7f0-7df9-4693-8cb4-454498d2facb None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:28.367205 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00070d2-f2ae-4bd8-82b0-13e0f3a7eb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:28.368069 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00070d2-f2ae-4bd8-82b0-13e0f3a7eb5f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:28.368324 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00070d2-f2ae-4bd8-82b0-13e0f3a7eb5f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:28.368609 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00070d2-f2ae-4bd8-82b0-13e0f3a7eb5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:28.396911 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beebafd5-7865-40bb-ad20-448453ec8656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:28.397766 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebafd5-7865-40bb-ad20-448453ec8656 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:28.398102 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebafd5-7865-40bb-ad20-448453ec8656 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:28.398380 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebafd5-7865-40bb-ad20-448453ec8656 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:28.415279 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beebafd5-7865-40bb-ad20-448453ec8656 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:28.416360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:28.417781 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beebafd5-7865-40bb-ad20-448453ec8656 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:28.447549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-beebafd5-7865-40bb-ad20-448453ec8656 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:28.447549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:28.447549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:28.447549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:28.447549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:28.447549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:28.447549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:28.447549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:28.447549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:28.447549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:28.447549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:28.447549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:28.447549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:09:28.447549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:09:28.447549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:09:28.448690 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:09:28.448690 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:28.448690 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:28.448690 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:28.448690 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:28.448690 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:28.448690 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:28.448690 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:28.448690 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:28.448690 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:28.448690 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:28.448690 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:28.448690 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:28.448690 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:28.448690 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:28.448690 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:28.448690 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:28.449827 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:09:28.449827 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:09:28.449827 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:28.449827 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:28.449827 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 416/848] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:09:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:28.450451 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beebafd5-7865-40bb-ad20-448453ec8656 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:28.481779 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00070d2-f2ae-4bd8-82b0-13e0f3a7eb5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:28.483394 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00070d2-f2ae-4bd8-82b0-13e0f3a7eb5f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:28.488844 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:28.627769 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beebafd5-7865-40bb-ad20-448453ec8656 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:28.662473 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 433/849] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:09:28 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 19:09:28.665036 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00070d2-f2ae-4bd8-82b0-13e0f3a7eb5f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:28.686949 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e5ae24-9354-4fa2-a9c8-329f2f1f0855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:28.687811 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5ae24-9354-4fa2-a9c8-329f2f1f0855 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:28.688157 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5ae24-9354-4fa2-a9c8-329f2f1f0855 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:28.688505 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5ae24-9354-4fa2-a9c8-329f2f1f0855 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:28.689331 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e5ae24-9354-4fa2-a9c8-329f2f1f0855 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:28.689716 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e5ae24-9354-4fa2-a9c8-329f2f1f0855 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:28.691393 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e5ae24-9354-4fa2-a9c8-329f2f1f0855 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:28.704486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 417/850] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Jul 25 19:09:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:28.704486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e5ae24-9354-4fa2-a9c8-329f2f1f0855 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:28.823890 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:28.847432 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:28.890369 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e5ae24-9354-4fa2-a9c8-329f2f1f0855 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:28.914315 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00070d2-f2ae-4bd8-82b0-13e0f3a7eb5f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:28.960283 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d3cfea-005c-4148-a21a-3690a200e0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:28.961719 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d3cfea-005c-4148-a21a-3690a200e0ea None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:28.962167 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d3cfea-005c-4148-a21a-3690a200e0ea None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:28.962651 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d3cfea-005c-4148-a21a-3690a200e0ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:28.988284 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d3cfea-005c-4148-a21a-3690a200e0ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:28.991702 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d3cfea-005c-4148-a21a-3690a200e0ea None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:28.996629 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc53d1cf-8ae5-4cd0-a937-a5ba62fd2766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:28.997863 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53d1cf-8ae5-4cd0-a937-a5ba62fd2766 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:28.998343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53d1cf-8ae5-4cd0-a937-a5ba62fd2766 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:28.998853 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53d1cf-8ae5-4cd0-a937-a5ba62fd2766 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:29.000330 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc53d1cf-8ae5-4cd0-a937-a5ba62fd2766 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:29.000853 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc53d1cf-8ae5-4cd0-a937-a5ba62fd2766 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:29.003680 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc53d1cf-8ae5-4cd0-a937-a5ba62fd2766 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:29.027463 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-13d3cfea-005c-4148-a21a-3690a200e0ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:29.027463 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:29.027463 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:29.027463 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:29.027463 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:29.027463 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:29.027463 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:29.027463 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:29.027463 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:29.027463 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:29.027463 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:29.027463 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:29.027463 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:09:29.027463 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:09:29.027463 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:09:29.028571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:09:29.028571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:29.028571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:29.028571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:29.028571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:29.028571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:29.028571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:29.028571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:29.028571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:29.028571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:29.028571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:29.028571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:29.028571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:29.028571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:29.028571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:29.028571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:29.028571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:29.029529 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:09:29.029529 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:09:29.029529 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:29.029529 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:29.029819 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 418/851] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:09:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:29.035810 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 434/852] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:09:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:29.036009 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d3cfea-005c-4148-a21a-3690a200e0ea None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:29.039339 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc53d1cf-8ae5-4cd0-a937-a5ba62fd2766 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:29.252890 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc53d1cf-8ae5-4cd0-a937-a5ba62fd2766 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:29.261714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d3cfea-005c-4148-a21a-3690a200e0ea None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:29.294171 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:29.312731 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062030c-4a58-410d-916e-da1dc575f064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:29.313810 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062030c-4a58-410d-916e-da1dc575f064 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:29.314261 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062030c-4a58-410d-916e-da1dc575f064 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:29.314372 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062030c-4a58-410d-916e-da1dc575f064 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:29.324035 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791ded7-b396-4653-834e-d4e7c3aedc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:29.325160 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791ded7-b396-4653-834e-d4e7c3aedc02 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:29.325620 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791ded7-b396-4653-834e-d4e7c3aedc02 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:29.326091 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791ded7-b396-4653-834e-d4e7c3aedc02 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:29.327321 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8791ded7-b396-4653-834e-d4e7c3aedc02 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:29.327845 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8791ded7-b396-4653-834e-d4e7c3aedc02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:29.329012 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062030c-4a58-410d-916e-da1dc575f064 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:29.330261 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8791ded7-b396-4653-834e-d4e7c3aedc02 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:29.331567 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062030c-4a58-410d-916e-da1dc575f064 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:29.332884 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:29.348812 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 419/853] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Jul 25 19:09:29 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:29.357583 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791ded7-b396-4653-834e-d4e7c3aedc02 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:29.358926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-f062030c-4a58-410d-916e-da1dc575f064 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 19:09:29.358926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:29.358926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:29.358926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:29.358926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:29.358926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:29.358926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:29.358926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:29.358926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:29.358926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:29.358926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:29.358926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:29.358926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:09:29.358926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:09:29.358926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:09:29.358926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:09:29.360904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:29.360904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:29.360904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:09:29.360904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:09:29.360904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:09:29.360904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:09:29.360904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:29.360904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:29.360904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:29.360904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:29.360904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:29.360904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:29.360904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:29.360904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:29.360904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:29.360904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:29.360904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:29.360904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:29.362353 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:29.362353 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:29.362353 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:09:29.362353 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:09:29.362353 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:09:29.362353 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:09:29.362353 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:09:29.362353 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:09:29.362353 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 19:09:29.362353 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:29.362353 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 435/854] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Jul 25 19:09:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:29.368004 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062030c-4a58-410d-916e-da1dc575f064 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:29.566422 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791ded7-b396-4653-834e-d4e7c3aedc02 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:29.584036 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062030c-4a58-410d-916e-da1dc575f064 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:29.644820 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc6558-3769-446b-ab32-b6b7f0178d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:29.646011 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc6558-3769-446b-ab32-b6b7f0178d51 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:29.646500 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc6558-3769-446b-ab32-b6b7f0178d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:29.647027 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc6558-3769-446b-ab32-b6b7f0178d51 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:29.648427 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dc6558-3769-446b-ab32-b6b7f0178d51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:29.648950 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dc6558-3769-446b-ab32-b6b7f0178d51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:29.652015 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dc6558-3769-446b-ab32-b6b7f0178d51 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:29.676253 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 420/855] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:09:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:29.676518 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5806db47-c73d-4942-9dc7-f6c154c7ce1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:29.677241 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806db47-c73d-4942-9dc7-f6c154c7ce1b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:29.677512 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806db47-c73d-4942-9dc7-f6c154c7ce1b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:29.679942 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806db47-c73d-4942-9dc7-f6c154c7ce1b None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:29.681317 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5806db47-c73d-4942-9dc7-f6c154c7ce1b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:29.681848 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5806db47-c73d-4942-9dc7-f6c154c7ce1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:29.684511 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5806db47-c73d-4942-9dc7-f6c154c7ce1b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:29.694797 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc6558-3769-446b-ab32-b6b7f0178d51 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:29.736452 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 436/856] 10.222.0.18 () {56 vars in 1135 bytes} [Mon Jul 25 19:09:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:29.747379 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5806db47-c73d-4942-9dc7-f6c154c7ce1b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:29.817761 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:29.828003 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:29.919583 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5806db47-c73d-4942-9dc7-f6c154c7ce1b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:29.926746 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc6558-3769-446b-ab32-b6b7f0178d51 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:29.970334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d755bc6a-d37c-49ba-bdcf-5a04c1cae859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:29.971379 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755bc6a-d37c-49ba-bdcf-5a04c1cae859 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:29.972052 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755bc6a-d37c-49ba-bdcf-5a04c1cae859 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:29.972549 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755bc6a-d37c-49ba-bdcf-5a04c1cae859 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:30.013976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f744984-f330-4e8e-a478-d104cf8859f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:30.015533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f744984-f330-4e8e-a478-d104cf8859f5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:30.016534 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f744984-f330-4e8e-a478-d104cf8859f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:30.017428 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f744984-f330-4e8e-a478-d104cf8859f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:30.038379 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f744984-f330-4e8e-a478-d104cf8859f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:30.041498 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f744984-f330-4e8e-a478-d104cf8859f5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:30.083847 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d755bc6a-d37c-49ba-bdcf-5a04c1cae859 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:30.085068 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-0f744984-f330-4e8e-a478-d104cf8859f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:09:30.085068 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:30.085068 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:30.085068 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:30.085068 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:30.085068 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:30.085068 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:30.085068 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:30.085068 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:30.085068 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:30.085068 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:30.085068 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:30.085068 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:09:30.085068 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:09:30.085068 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:09:30.086131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:09:30.086131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:30.086131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:30.086131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:30.086131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:30.086131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:30.086131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:30.086131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:30.086131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:30.086131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:30.086131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:30.086131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:30.086131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:30.086131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:30.086131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:30.086131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:30.086131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:30.087191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:09:30.087191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:09:30.087191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:09:30.087191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:09:30.087191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:09:30.087191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:09:30.087191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:09:30.087191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:30.095602 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d755bc6a-d37c-49ba-bdcf-5a04c1cae859 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:30.095803 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 421/857] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:09:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 403 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:09:30.100173 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f744984-f330-4e8e-a478-d104cf8859f5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:30.180823 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:30.280811 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 437/858] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:09:29 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 19:09:30.282233 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d755bc6a-d37c-49ba-bdcf-5a04c1cae859 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:30.326732 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f744984-f330-4e8e-a478-d104cf8859f5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:30.364459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:30.397967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0a067-2943-4899-bbd6-24b6dfa1be98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:30.398913 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0a067-2943-4899-bbd6-24b6dfa1be98 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:30.399333 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0a067-2943-4899-bbd6-24b6dfa1be98 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:30.399778 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0a067-2943-4899-bbd6-24b6dfa1be98 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:30.401753 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf0a067-2943-4899-bbd6-24b6dfa1be98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:30.402791 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf0a067-2943-4899-bbd6-24b6dfa1be98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:30.405066 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf0a067-2943-4899-bbd6-24b6dfa1be98 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:30.427331 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 422/859] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Jul 25 19:09:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:30.434490 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44288ab6-bf63-4a67-86c9-8f33c6bb1c7e None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:30.434846 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44288ab6-bf63-4a67-86c9-8f33c6bb1c7e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:30.435073 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44288ab6-bf63-4a67-86c9-8f33c6bb1c7e None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:30.436526 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 423/860] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:30.437313 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44288ab6-bf63-4a67-86c9-8f33c6bb1c7e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:30.445285 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d755bc6a-d37c-49ba-bdcf-5a04c1cae859 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:30.491719 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d151ae5-b8c4-442d-bce8-53f9715370aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:30.492458 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d151ae5-b8c4-442d-bce8-53f9715370aa None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:30.492758 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d151ae5-b8c4-442d-bce8-53f9715370aa None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:30.493045 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d151ae5-b8c4-442d-bce8-53f9715370aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:30.507374 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d151ae5-b8c4-442d-bce8-53f9715370aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:30.508900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d151ae5-b8c4-442d-bce8-53f9715370aa None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:30.512282 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:30.532835 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-8d151ae5-b8c4-442d-bce8-53f9715370aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:30.532835 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:30.532835 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:30.532835 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:30.532835 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:30.532835 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:30.532835 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:30.532835 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:30.532835 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:30.532835 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:30.532835 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:30.532835 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:30.532835 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:09:30.532835 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:09:30.532835 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:09:30.533823 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:09:30.533823 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:30.533823 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:30.533823 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:30.533823 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:30.533823 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:30.533823 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:30.533823 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:30.533823 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:30.533823 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:30.533823 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:30.533823 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:30.533823 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:30.533823 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:30.533823 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:30.533823 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:30.533823 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:30.534854 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:09:30.534854 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:09:30.534854 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:30.534854 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:30.534854 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 438/861] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:09:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:30.535708 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ca8f1-68f5-4b4b-b01a-c10a616f164d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:30.535708 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ca8f1-68f5-4b4b-b01a-c10a616f164d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:30.535985 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ca8f1-68f5-4b4b-b01a-c10a616f164d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:30.559583 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-904ca8f1-68f5-4b4b-b01a-c10a616f164d None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:30.560863 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-904ca8f1-68f5-4b4b-b01a-c10a616f164d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:30.641476 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44288ab6-bf63-4a67-86c9-8f33c6bb1c7e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:30.684191 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 439/862] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:30 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:09:30.685583 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ca8f1-68f5-4b4b-b01a-c10a616f164d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:30.707334 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb286b-028b-4576-9ac2-eff954df8466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:30.709279 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb286b-028b-4576-9ac2-eff954df8466 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:30.710724 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb286b-028b-4576-9ac2-eff954df8466 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:30.711595 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb286b-028b-4576-9ac2-eff954df8466 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:30.714171 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb286b-028b-4576-9ac2-eff954df8466 None admin] RBAC: Adding query filter params (role.id=23718d83b0864b16b0071e8ffe0b5861, scope.project.id=191069caccd04233b012585e29cfa18b, user.id=1c0772205ae24713b8901666a87018b2) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:30.715727 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb286b-028b-4576-9ac2-eff954df8466 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:30.720147 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb286b-028b-4576-9ac2-eff954df8466 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:30.734976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 424/863] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:09:30 2022] GET /identity/v3/role_assignments?user.id=1c0772205ae24713b8901666a87018b2&scope.project.id=191069caccd04233b012585e29cfa18b&role.id=23718d83b0864b16b0071e8ffe0b5861 => generated 608 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:30.736912 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb286b-028b-4576-9ac2-eff954df8466 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:30.813701 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:30.888925 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ca8f1-68f5-4b4b-b01a-c10a616f164d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:30.927121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:30.943179 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb286b-028b-4576-9ac2-eff954df8466 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:30.945886 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae967b-9931-44c5-bdba-482f04c76858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:30.946593 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae967b-9931-44c5-bdba-482f04c76858 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:30.946882 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae967b-9931-44c5-bdba-482f04c76858 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:30.947163 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae967b-9931-44c5-bdba-482f04c76858 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:30.948718 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae967b-9931-44c5-bdba-482f04c76858 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:30.949379 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae967b-9931-44c5-bdba-482f04c76858 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:30.952194 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae967b-9931-44c5-bdba-482f04c76858 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:30.973768 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 440/864] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:09:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:30.975553 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae967b-9931-44c5-bdba-482f04c76858 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:31.017468 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac45ea-8fb1-4d30-a57e-3a71503645c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:31.018591 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac45ea-8fb1-4d30-a57e-3a71503645c2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:31.018893 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac45ea-8fb1-4d30-a57e-3a71503645c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:31.019246 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac45ea-8fb1-4d30-a57e-3a71503645c2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:31.019961 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ac45ea-8fb1-4d30-a57e-3a71503645c2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:31.021408 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ac45ea-8fb1-4d30-a57e-3a71503645c2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:31.041664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-78ac45ea-8fb1-4d30-a57e-3a71503645c2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 19:09:31.041664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:31.041664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 19:09:31.041664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 19:09:31.041664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 19:09:31.041664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 19:09:31.041664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 19:09:31.041664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:31.041664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 19:09:31.041664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:31.041664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 19:09:31.041664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:31.041664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:31.041664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:31.041664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:31.041664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:31.043465 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:31.043465 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:31.043465 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:31.043465 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:31.043465 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:31.043465 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:31.043465 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:31.043465 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 19:09:31.043465 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 19:09:31.043465 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 19:09:31.043465 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 19:09:31.043465 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:31.043465 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:31.043465 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:31.043465 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:31.043465 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:31.045282 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:31.045282 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:31.045282 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:31.045282 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:31.045282 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:31.045282 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:31.045282 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:31.045282 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:31.045282 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:31.045282 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:31.045282 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:31.045282 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 19:09:31.045282 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:31.045282 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 19:09:31.045282 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:31.045282 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 425/865] 10.222.0.18 () {56 vars in 1132 bytes} [Mon Jul 25 19:09:30 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:31.053748 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac45ea-8fb1-4d30-a57e-3a71503645c2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:31.205888 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae967b-9931-44c5-bdba-482f04c76858 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:31.208939 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac45ea-8fb1-4d30-a57e-3a71503645c2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:31.218543 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:31.262573 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef93b8c-53ed-4cdd-af27-45adab73f008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:31.262573 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef93b8c-53ed-4cdd-af27-45adab73f008 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:31.262573 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef93b8c-53ed-4cdd-af27-45adab73f008 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:31.262573 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef93b8c-53ed-4cdd-af27-45adab73f008 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:31.263257 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef93b8c-53ed-4cdd-af27-45adab73f008 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:31.265729 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef93b8c-53ed-4cdd-af27-45adab73f008 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:31.265729 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef93b8c-53ed-4cdd-af27-45adab73f008 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:31.277956 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c3675-f517-4cb9-9835-0b7ea8cde3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:31.279397 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c3675-f517-4cb9-9835-0b7ea8cde3cf None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:31.279921 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c3675-f517-4cb9-9835-0b7ea8cde3cf None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:31.280544 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c3675-f517-4cb9-9835-0b7ea8cde3cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:31.285171 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 426/866] 10.222.0.18 () {56 vars in 1126 bytes} [Mon Jul 25 19:09:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:31.295653 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef93b8c-53ed-4cdd-af27-45adab73f008 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:31.295860 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92c3675-f517-4cb9-9835-0b7ea8cde3cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:31.298090 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92c3675-f517-4cb9-9835-0b7ea8cde3cf None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:31.326764 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e92c3675-f517-4cb9-9835-0b7ea8cde3cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:09:31.326764 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:31.326764 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:31.326764 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:31.326764 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:31.326764 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:31.326764 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:31.326764 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:31.326764 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:31.326764 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:31.326764 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:31.326764 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:31.326764 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:09:31.326764 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:09:31.326764 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:09:31.328258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:09:31.328258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:31.328258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:31.328258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:31.328258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:31.328258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:31.328258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:31.328258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:31.328258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:31.328258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:31.328258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:31.328258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:31.328258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:31.328258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:31.328258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:31.328258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:31.328258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:31.329251 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:09:31.329251 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:09:31.329251 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:09:31.329251 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:09:31.329251 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:09:31.329251 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:09:31.329251 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:09:31.329251 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:31.335735 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 441/867] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:09:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:31.340294 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c3675-f517-4cb9-9835-0b7ea8cde3cf None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:31.435077 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:31.540100 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef93b8c-53ed-4cdd-af27-45adab73f008 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:31.552054 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c3675-f517-4cb9-9835-0b7ea8cde3cf None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:31.620265 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1bd9f-e2ec-4454-96cf-68bbf593cc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:31.621382 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63095233-e3b8-45df-8384-a582c01f6ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:31.622437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63095233-e3b8-45df-8384-a582c01f6ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:31.622616 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1bd9f-e2ec-4454-96cf-68bbf593cc6e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:31.622910 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63095233-e3b8-45df-8384-a582c01f6ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:31.623394 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63095233-e3b8-45df-8384-a582c01f6ac3 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:31.624005 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1bd9f-e2ec-4454-96cf-68bbf593cc6e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:31.624749 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1bd9f-e2ec-4454-96cf-68bbf593cc6e None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:31.624921 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63095233-e3b8-45df-8384-a582c01f6ac3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:31.625404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63095233-e3b8-45df-8384-a582c01f6ac3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:31.626733 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f1bd9f-e2ec-4454-96cf-68bbf593cc6e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:31.627567 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f1bd9f-e2ec-4454-96cf-68bbf593cc6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:31.628063 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63095233-e3b8-45df-8384-a582c01f6ac3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:31.630652 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f1bd9f-e2ec-4454-96cf-68bbf593cc6e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:31.641153 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 427/868] 10.222.0.18 () {56 vars in 1126 bytes} [Mon Jul 25 19:09:31 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:31.655667 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 442/869] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Jul 25 19:09:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:31.658751 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1bd9f-e2ec-4454-96cf-68bbf593cc6e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:31.743000 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:31.891568 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1bd9f-e2ec-4454-96cf-68bbf593cc6e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:31.965080 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbef61a-7d54-464b-9cc4-d92b946df7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:31.966532 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbef61a-7d54-464b-9cc4-d92b946df7d1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:31.971550 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbef61a-7d54-464b-9cc4-d92b946df7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:31.971550 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbef61a-7d54-464b-9cc4-d92b946df7d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:31.971550 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbef61a-7d54-464b-9cc4-d92b946df7d1 None admin] RBAC: Adding query filter params (role.id=ed7fec18968241efa76a0ee769d96da9, scope.project.id=191069caccd04233b012585e29cfa18b, user.id=feafcf1b5a9844fe9738823fddd07e0e) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:31.971550 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbef61a-7d54-464b-9cc4-d92b946df7d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:31.972602 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbef61a-7d54-464b-9cc4-d92b946df7d1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:31.986139 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 428/870] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:09:31 2022] GET /identity/v3/role_assignments?user.id=feafcf1b5a9844fe9738823fddd07e0e&scope.project.id=191069caccd04233b012585e29cfa18b&role.id=ed7fec18968241efa76a0ee769d96da9 => generated 608 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:09:32.781955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb469c9-e2b3-4f21-b3c1-d1fbf5369466 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:32.782769 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb469c9-e2b3-4f21-b3c1-d1fbf5369466 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:32.782906 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb469c9-e2b3-4f21-b3c1-d1fbf5369466 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:32.786041 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 443/871] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:32.795712 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86cfa9-fd9b-4839-b63a-8f3ba859be7e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:32.796173 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86cfa9-fd9b-4839-b63a-8f3ba859be7e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:32.796821 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86cfa9-fd9b-4839-b63a-8f3ba859be7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:32.830494 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4a86cfa9-fd9b-4839-b63a-8f3ba859be7e None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:32.832664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4a86cfa9-fd9b-4839-b63a-8f3ba859be7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:32.975536 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 429/872] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:09:33.008683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb469c9-e2b3-4f21-b3c1-d1fbf5369466 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:33.229038 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb469c9-e2b3-4f21-b3c1-d1fbf5369466 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:33.234670 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:33.309410 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ea6f77-9a28-4908-8d12-76fc86e90c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:33.310772 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea6f77-9a28-4908-8d12-76fc86e90c5d None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:33.311247 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea6f77-9a28-4908-8d12-76fc86e90c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:33.311816 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ea6f77-9a28-4908-8d12-76fc86e90c5d None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:33.313244 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ea6f77-9a28-4908-8d12-76fc86e90c5d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:33.316112 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ea6f77-9a28-4908-8d12-76fc86e90c5d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:33.355757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 444/873] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Jul 25 19:09:33 2022] POST /identity/v3/services => generated 251 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:09:33.931690 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68b0a1-3aa2-4016-acba-c65d4547d27f None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:33.932842 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68b0a1-3aa2-4016-acba-c65d4547d27f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:33.933406 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68b0a1-3aa2-4016-acba-c65d4547d27f None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:33.936839 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 430/874] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:33.946294 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1bf850-efd1-4660-b7c1-572fcb188ac6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:33.946963 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1bf850-efd1-4660-b7c1-572fcb188ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:33.947657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1bf850-efd1-4660-b7c1-572fcb188ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:33.981957 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2f1bf850-efd1-4660-b7c1-572fcb188ac6 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:33.984471 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2f1bf850-efd1-4660-b7c1-572fcb188ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:34.097161 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 445/875] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:33 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:09:34.147911 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68b0a1-3aa2-4016-acba-c65d4547d27f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:34.194157 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:34.305260 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68b0a1-3aa2-4016-acba-c65d4547d27f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:34.354988 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da7e48e-2b0d-4065-ab6b-a6469ecc25c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:34.356014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7e48e-2b0d-4065-ab6b-a6469ecc25c5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:34.356299 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7e48e-2b0d-4065-ab6b-a6469ecc25c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:34.356572 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7e48e-2b0d-4065-ab6b-a6469ecc25c5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:34.375438 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da7e48e-2b0d-4065-ab6b-a6469ecc25c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:09:34.380035 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccd760-d850-4c6f-a810-bec5d48f7621 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:34.380807 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccd760-d850-4c6f-a810-bec5d48f7621 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:34.381380 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccd760-d850-4c6f-a810-bec5d48f7621 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:34.383582 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 446/876] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:09:34.385762 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-3da7e48e-2b0d-4065-ab6b-a6469ecc25c5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 19:09:34.385762 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:34.385762 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:34.385762 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:34.385762 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:34.385762 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:34.385762 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:34.385762 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:34.385762 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:34.385762 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:34.385762 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:34.385762 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:34.385762 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:09:34.385762 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:09:34.385762 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:09:34.385762 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:09:34.386830 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:09:34.386830 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:09:34.386830 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:09:34.386830 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:09:34.386830 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:34.386830 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:34.386830 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:34.386830 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:34.386830 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:34.386830 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:34.386830 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:34.386830 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:34.386830 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:34.386830 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:34.386830 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:34.386830 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:34.388294 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:34.388294 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:34.388294 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:34.388294 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:34.388294 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:09:34.388294 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:09:34.388294 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:09:34.388294 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:09:34.388294 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:09:34.388294 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:09:34.388294 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 19:09:34.388294 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:34.388294 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 431/877] 10.222.0.18 () {56 vars in 1120 bytes} [Mon Jul 25 19:09:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:34.393626 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa0749-6c33-4159-acca-463b96eec269 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:34.394360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa0749-6c33-4159-acca-463b96eec269 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:34.394857 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa0749-6c33-4159-acca-463b96eec269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:34.397212 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da7e48e-2b0d-4065-ab6b-a6469ecc25c5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:34.422440 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-78aa0749-6c33-4159-acca-463b96eec269 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:34.425174 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-78aa0749-6c33-4159-acca-463b96eec269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:34.531831 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 447/878] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:34 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:09:34.556457 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa0749-6c33-4159-acca-463b96eec269 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:34.557573 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:34.575270 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da7e48e-2b0d-4065-ab6b-a6469ecc25c5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:34.586170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:34.628368 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b58931-de61-4331-be93-63dd37761f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:34.629346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b58931-de61-4331-be93-63dd37761f91 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:34.629814 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b58931-de61-4331-be93-63dd37761f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:34.630258 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b58931-de61-4331-be93-63dd37761f91 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:34.631318 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b58931-de61-4331-be93-63dd37761f91 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:34.631851 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b58931-de61-4331-be93-63dd37761f91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:34.633476 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b58931-de61-4331-be93-63dd37761f91 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:34.651871 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 432/879] 10.222.0.18 () {56 vars in 1115 bytes} [Mon Jul 25 19:09:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:34.654219 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b58931-de61-4331-be93-63dd37761f91 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:34.715543 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa0749-6c33-4159-acca-463b96eec269 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:34.772562 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df271d7-6008-4355-9100-277fb48865f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:34.773085 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df271d7-6008-4355-9100-277fb48865f9 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:34.773433 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df271d7-6008-4355-9100-277fb48865f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:34.773786 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df271d7-6008-4355-9100-277fb48865f9 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:34.774706 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df271d7-6008-4355-9100-277fb48865f9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:34.778654 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df271d7-6008-4355-9100-277fb48865f9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:34.803562 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 448/880] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Jul 25 19:09:34 2022] POST /identity/v3/services => generated 269 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:09:34.817163 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b58931-de61-4331-be93-63dd37761f91 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:34.882486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec95cb-20b2-4a63-86bb-9cdf22fa1b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:34.883930 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec95cb-20b2-4a63-86bb-9cdf22fa1b51 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:34.884575 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec95cb-20b2-4a63-86bb-9cdf22fa1b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:34.885025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec95cb-20b2-4a63-86bb-9cdf22fa1b51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:34.904299 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec95cb-20b2-4a63-86bb-9cdf22fa1b51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:34.906406 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec95cb-20b2-4a63-86bb-9cdf22fa1b51 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:34.917862 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:34.934507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-36ec95cb-20b2-4a63-86bb-9cdf22fa1b51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:34.934507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:34.934507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:34.934507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:34.934507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:34.934507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:34.934507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:34.934507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:34.934507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:34.934507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:34.934507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:34.934507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:34.934507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:09:34.934507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:09:34.934507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:09:34.935918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:09:34.935918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:34.935918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:34.935918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:34.935918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:34.935918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:34.935918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:34.935918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:34.935918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:34.935918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:34.935918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:34.935918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:34.935918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:34.935918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:34.935918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:34.935918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:34.935918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:34.937036 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:09:34.937036 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:09:34.937036 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:34.937036 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:34.939322 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 433/881] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:09:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:09:34.946229 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df271d7-6008-4355-9100-277fb48865f9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:35.074736 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:35.113430 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df271d7-6008-4355-9100-277fb48865f9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:35.181022 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d198b73f-eff3-49c4-9104-517abf921b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:35.182807 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d198b73f-eff3-49c4-9104-517abf921b5e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:35.183451 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d198b73f-eff3-49c4-9104-517abf921b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:35.183451 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d198b73f-eff3-49c4-9104-517abf921b5e None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:35.186570 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d198b73f-eff3-49c4-9104-517abf921b5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:35.186570 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d198b73f-eff3-49c4-9104-517abf921b5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:35.188487 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d198b73f-eff3-49c4-9104-517abf921b5e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:35.201371 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7528a0a-5dbc-4bf8-9006-bf3bccb79baf None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:35.202085 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7528a0a-5dbc-4bf8-9006-bf3bccb79baf None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:35.202085 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7528a0a-5dbc-4bf8-9006-bf3bccb79baf None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:35.204079 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 434/882] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:09:35.210122 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79e30c-45ad-4121-b48b-725907318f3f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:35.210469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79e30c-45ad-4121-b48b-725907318f3f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:35.210686 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79e30c-45ad-4121-b48b-725907318f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:35.213929 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 449/883] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:09:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:35.221489 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d198b73f-eff3-49c4-9104-517abf921b5e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:35.249027 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2f79e30c-45ad-4121-b48b-725907318f3f None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:35.251757 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2f79e30c-45ad-4121-b48b-725907318f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:35.364189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 435/884] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:09:35.387491 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79e30c-45ad-4121-b48b-725907318f3f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:35.435142 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d198b73f-eff3-49c4-9104-517abf921b5e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:35.499156 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac706d-2754-4607-8c01-85058887d284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:35.501063 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac706d-2754-4607-8c01-85058887d284 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:35.502054 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac706d-2754-4607-8c01-85058887d284 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:35.502343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac706d-2754-4607-8c01-85058887d284 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:35.516980 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac706d-2754-4607-8c01-85058887d284 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:35.518857 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac706d-2754-4607-8c01-85058887d284 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:35.522656 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:35.544215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-04ac706d-2754-4607-8c01-85058887d284 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 19:09:35.544215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:35.544215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:35.544215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:35.544215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:35.544215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:35.544215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:35.544215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:35.544215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:35.544215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:35.544215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:35.544215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:35.544215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:09:35.544215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:09:35.544215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:09:35.544215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:09:35.545263 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:35.545263 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:35.545263 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:09:35.545263 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:09:35.545263 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:09:35.545263 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:09:35.545263 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:35.545263 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:35.545263 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:35.545263 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:35.545263 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:35.545263 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:35.545263 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:35.545263 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:35.545263 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:35.545263 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:35.545263 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:35.545263 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:35.546334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:35.546334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:35.546334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:09:35.546334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:09:35.546334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:09:35.546334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:09:35.546334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:09:35.546334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:09:35.546334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 19:09:35.546334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:35.554761 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 450/885] 10.222.0.18 () {56 vars in 1117 bytes} [Mon Jul 25 19:09:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:09:35.557266 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac706d-2754-4607-8c01-85058887d284 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:35.591593 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79e30c-45ad-4121-b48b-725907318f3f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:35.633988 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:35.645784 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0890a587-47ad-4d3a-86ff-e7fa851a5c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:35.646918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890a587-47ad-4d3a-86ff-e7fa851a5c23 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:35.648087 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890a587-47ad-4d3a-86ff-e7fa851a5c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:35.648604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890a587-47ad-4d3a-86ff-e7fa851a5c23 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:35.649940 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0890a587-47ad-4d3a-86ff-e7fa851a5c23 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:35.656232 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0890a587-47ad-4d3a-86ff-e7fa851a5c23 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:35.683126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-0890a587-47ad-4d3a-86ff-e7fa851a5c23 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 19:09:35.683126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:35.683126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 19:09:35.683126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 19:09:35.683126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 19:09:35.683126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 19:09:35.683126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 19:09:35.683126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:35.683126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 19:09:35.683126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:35.683126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 19:09:35.683126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:35.683126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:35.683126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:35.683126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:35.683126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:35.683126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:35.685214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:35.685214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:35.685214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:35.685214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:35.685214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:35.685214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:35.685214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 19:09:35.685214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 19:09:35.685214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 19:09:35.685214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 19:09:35.685214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:35.685214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:35.685214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:35.685214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:35.685214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:35.685214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:35.686869 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:35.686869 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:35.686869 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:35.686869 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:35.686869 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:35.686869 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:35.686869 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:35.686869 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:35.686869 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:35.686869 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:35.686869 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 19:09:35.686869 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:35.686869 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 19:09:35.686869 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:35.692543 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 436/886] 10.222.0.18 () {56 vars in 1126 bytes} [Mon Jul 25 19:09:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:35.699572 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0890a587-47ad-4d3a-86ff-e7fa851a5c23 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:35.736052 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac706d-2754-4607-8c01-85058887d284 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:35.786600 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0167beb2-8010-4e79-829a-8b931c2099af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:35.788149 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167beb2-8010-4e79-829a-8b931c2099af None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:35.788694 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167beb2-8010-4e79-829a-8b931c2099af None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:35.789205 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167beb2-8010-4e79-829a-8b931c2099af None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:35.790456 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0167beb2-8010-4e79-829a-8b931c2099af None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:35.791040 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0167beb2-8010-4e79-829a-8b931c2099af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:35.793799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0167beb2-8010-4e79-829a-8b931c2099af None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:35.816060 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 451/887] 10.222.0.18 () {56 vars in 1149 bytes} [Mon Jul 25 19:09:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:35.823910 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0167beb2-8010-4e79-829a-8b931c2099af None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:35.859102 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0890a587-47ad-4d3a-86ff-e7fa851a5c23 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:35.904900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced428a2-3f61-4e09-8cb1-795603db84eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:35.906149 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced428a2-3f61-4e09-8cb1-795603db84eb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:35.906748 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced428a2-3f61-4e09-8cb1-795603db84eb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:35.907308 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced428a2-3f61-4e09-8cb1-795603db84eb None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:35.908339 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced428a2-3f61-4e09-8cb1-795603db84eb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:35.908902 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced428a2-3f61-4e09-8cb1-795603db84eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:35.910370 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:35.912161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced428a2-3f61-4e09-8cb1-795603db84eb None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:35.928159 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 437/888] 10.222.0.18 () {56 vars in 1122 bytes} [Mon Jul 25 19:09:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:09:35.930639 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced428a2-3f61-4e09-8cb1-795603db84eb None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:35.967459 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0167beb2-8010-4e79-829a-8b931c2099af None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:36.005992 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:36.016444 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58b531-5077-4fab-87c2-96dac8a8985b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:36.017147 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58b531-5077-4fab-87c2-96dac8a8985b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:36.017438 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58b531-5077-4fab-87c2-96dac8a8985b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:36.017716 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58b531-5077-4fab-87c2-96dac8a8985b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:36.072070 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced428a2-3f61-4e09-8cb1-795603db84eb None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:36.108096 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d58b531-5077-4fab-87c2-96dac8a8985b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:36.109368 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d58b531-5077-4fab-87c2-96dac8a8985b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:36.117698 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ad730-269f-4db3-8d75-7b7190f514cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:36.118932 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ad730-269f-4db3-8d75-7b7190f514cc None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:36.119437 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ad730-269f-4db3-8d75-7b7190f514cc None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:36.120862 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ad730-269f-4db3-8d75-7b7190f514cc None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:36.121905 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ad730-269f-4db3-8d75-7b7190f514cc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:36.122607 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ad730-269f-4db3-8d75-7b7190f514cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:36.124485 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ad730-269f-4db3-8d75-7b7190f514cc None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:36.136313 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 438/889] 10.222.0.18 () {56 vars in 1122 bytes} [Mon Jul 25 19:09:35 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:09:36.194191 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:36.255875 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 452/890] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:09:35 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 19:09:36.268933 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ad730-269f-4db3-8d75-7b7190f514cc None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:36.413741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ad730-269f-4db3-8d75-7b7190f514cc None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:36.442068 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:36.462103 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e84e1-8e00-473c-8b7b-44ba6b3ddc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:36.462941 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e84e1-8e00-473c-8b7b-44ba6b3ddc75 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:36.463888 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e84e1-8e00-473c-8b7b-44ba6b3ddc75 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:36.463888 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e84e1-8e00-473c-8b7b-44ba6b3ddc75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:36.464760 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cab3fd3-8962-46de-b704-144fb0f953eb None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:36.465065 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cab3fd3-8962-46de-b704-144fb0f953eb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:36.465353 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cab3fd3-8962-46de-b704-144fb0f953eb None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:36.467328 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 453/891] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:09:36.474300 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8562d-fe35-43da-a333-d7895f7d525f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:36.475815 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8562d-fe35-43da-a333-d7895f7d525f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:36.476779 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8562d-fe35-43da-a333-d7895f7d525f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:36.485722 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4e84e1-8e00-473c-8b7b-44ba6b3ddc75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:36.487260 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4e84e1-8e00-473c-8b7b-44ba6b3ddc75 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:36.518437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-eeb8562d-fe35-43da-a333-d7895f7d525f None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:36.519442 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-6d4e84e1-8e00-473c-8b7b-44ba6b3ddc75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:36.519442 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:36.519442 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:36.519442 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:36.519442 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:36.519442 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:36.519442 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:36.519442 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:36.519442 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:36.519442 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:36.519442 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:36.519442 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:36.519442 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:09:36.519442 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:09:36.519442 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:09:36.521180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:09:36.521180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:36.521180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:36.521180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:36.521180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:36.521180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:36.521180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:36.521180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:36.521180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:36.521180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:36.521180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:36.521180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:36.521180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:36.521180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:36.521180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:36.521180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:36.521180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:36.522860 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:09:36.522860 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:09:36.522860 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:36.522860 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:36.523349 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-eeb8562d-fe35-43da-a333-d7895f7d525f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:36.523494 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 439/892] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:09:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:36.531174 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e84e1-8e00-473c-8b7b-44ba6b3ddc75 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:36.678211 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 454/893] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:09:36.704791 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb8562d-fe35-43da-a333-d7895f7d525f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:36.742801 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e84e1-8e00-473c-8b7b-44ba6b3ddc75 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:36.825027 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4da43-5775-4a9d-878c-de36cc0a8cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:36.826203 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4da43-5775-4a9d-878c-de36cc0a8cc3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:36.826826 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4da43-5775-4a9d-878c-de36cc0a8cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:36.827302 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4da43-5775-4a9d-878c-de36cc0a8cc3 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:36.828880 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb4da43-5775-4a9d-878c-de36cc0a8cc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:36.829446 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb4da43-5775-4a9d-878c-de36cc0a8cc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:36.832384 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb4da43-5775-4a9d-878c-de36cc0a8cc3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:36.841005 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:36.859744 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 440/894] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:36.861706 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb8562d-fe35-43da-a333-d7895f7d525f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:36.868504 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4da43-5775-4a9d-878c-de36cc0a8cc3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:36.917397 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6310520-218e-4340-ae78-dbcd882d95d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:36.918406 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6310520-218e-4340-ae78-dbcd882d95d2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:36.918700 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6310520-218e-4340-ae78-dbcd882d95d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:36.918970 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6310520-218e-4340-ae78-dbcd882d95d2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:36.920592 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6310520-218e-4340-ae78-dbcd882d95d2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:36.922075 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6310520-218e-4340-ae78-dbcd882d95d2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:36.939946 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:36.947213 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d6310520-218e-4340-ae78-dbcd882d95d2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 19:09:36.947213 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:36.947213 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 19:09:36.947213 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 19:09:36.947213 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 19:09:36.947213 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 19:09:36.947213 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 19:09:36.947213 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:36.947213 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 19:09:36.947213 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:36.947213 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 19:09:36.947213 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:36.947213 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:36.947213 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:36.947213 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:36.947213 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:36.947213 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:36.949958 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:36.949958 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:36.949958 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:36.949958 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:36.949958 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:36.949958 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:36.949958 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 19:09:36.949958 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 19:09:36.949958 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 19:09:36.949958 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 19:09:36.949958 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:36.949958 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:36.949958 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:36.949958 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:36.949958 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:36.949958 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:36.951920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:36.951920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:36.951920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:36.951920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:36.951920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:36.951920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:36.951920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:36.951920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:36.951920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:36.951920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:36.951920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 19:09:36.951920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:36.951920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 19:09:36.951920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:36.958459 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 455/895] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Jul 25 19:09:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:09:36.968842 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6310520-218e-4340-ae78-dbcd882d95d2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:37.040708 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4da43-5775-4a9d-878c-de36cc0a8cc3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:37.084866 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb29854d-83d3-47cf-925b-1e6b2b0ecd8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:37.086077 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29854d-83d3-47cf-925b-1e6b2b0ecd8a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:37.086437 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29854d-83d3-47cf-925b-1e6b2b0ecd8a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:37.086786 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29854d-83d3-47cf-925b-1e6b2b0ecd8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:37.096035 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb29854d-83d3-47cf-925b-1e6b2b0ecd8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:37.096455 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6310520-218e-4340-ae78-dbcd882d95d2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:37.097714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb29854d-83d3-47cf-925b-1e6b2b0ecd8a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:37.116038 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-eb29854d-83d3-47cf-925b-1e6b2b0ecd8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:09:37.116038 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:37.116038 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:37.116038 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:37.116038 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:37.116038 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:37.116038 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:37.116038 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:37.116038 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:37.116038 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:37.116038 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:37.116038 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:37.116038 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:09:37.116038 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:09:37.116038 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:09:37.119047 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:09:37.119047 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:37.119047 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:37.119047 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:37.119047 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:37.119047 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:37.119047 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:37.119047 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:37.119047 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:37.119047 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:37.119047 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:37.119047 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:37.119047 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:37.119047 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:37.119047 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:37.119047 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:37.119047 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:37.122970 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:09:37.122970 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:09:37.122970 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:09:37.122970 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:09:37.122970 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:09:37.122970 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:09:37.122970 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:09:37.122970 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:37.122970 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 441/896] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:09:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:09:37.135564 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb29854d-83d3-47cf-925b-1e6b2b0ecd8a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:37.141035 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8bcf7-69a4-4a66-bc24-7975a3d4fa99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:37.142381 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8bcf7-69a4-4a66-bc24-7975a3d4fa99 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:37.142973 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8bcf7-69a4-4a66-bc24-7975a3d4fa99 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:37.143608 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8bcf7-69a4-4a66-bc24-7975a3d4fa99 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:37.145129 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf8bcf7-69a4-4a66-bc24-7975a3d4fa99 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:37.145771 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf8bcf7-69a4-4a66-bc24-7975a3d4fa99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:37.148576 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf8bcf7-69a4-4a66-bc24-7975a3d4fa99 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:37.163235 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 456/897] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Jul 25 19:09:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:37.168656 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8bcf7-69a4-4a66-bc24-7975a3d4fa99 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:37.208113 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:37.292528 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:37.308522 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb29854d-83d3-47cf-925b-1e6b2b0ecd8a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:37.356307 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8bcf7-69a4-4a66-bc24-7975a3d4fa99 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:37.362358 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0a85d-2eb9-414c-8cb3-8b2693c2eb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:37.363156 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0a85d-2eb9-414c-8cb3-8b2693c2eb7c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:37.363456 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0a85d-2eb9-414c-8cb3-8b2693c2eb7c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:37.363765 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0a85d-2eb9-414c-8cb3-8b2693c2eb7c None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:37.364593 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0a85d-2eb9-414c-8cb3-8b2693c2eb7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:37.365052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0a85d-2eb9-414c-8cb3-8b2693c2eb7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:37.366690 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0a85d-2eb9-414c-8cb3-8b2693c2eb7c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:37.391901 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 442/898] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Jul 25 19:09:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:09:37.394933 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0a85d-2eb9-414c-8cb3-8b2693c2eb7c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:37.417899 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7183cc27-a016-4435-96e5-82dc84c48fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:37.419607 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183cc27-a016-4435-96e5-82dc84c48fe8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:37.420180 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183cc27-a016-4435-96e5-82dc84c48fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:37.421327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183cc27-a016-4435-96e5-82dc84c48fe8 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:37.422661 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7183cc27-a016-4435-96e5-82dc84c48fe8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:37.423237 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7183cc27-a016-4435-96e5-82dc84c48fe8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:37.426063 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7183cc27-a016-4435-96e5-82dc84c48fe8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:37.441881 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 457/899] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Jul 25 19:09:37 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:37.442449 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14334dec-33fe-4f08-a591-0aecbc277706 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:37.442753 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14334dec-33fe-4f08-a591-0aecbc277706 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:37.443388 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14334dec-33fe-4f08-a591-0aecbc277706 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:37.444675 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 458/900] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:09:37.445683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14334dec-33fe-4f08-a591-0aecbc277706 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:37.542034 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0a85d-2eb9-414c-8cb3-8b2693c2eb7c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:37.561439 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:37.594309 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b255a-f914-440e-9577-ae72370bde74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:37.595437 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b255a-f914-440e-9577-ae72370bde74 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:37.595975 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b255a-f914-440e-9577-ae72370bde74 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:37.596289 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14334dec-33fe-4f08-a591-0aecbc277706 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:37.596435 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b255a-f914-440e-9577-ae72370bde74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:37.597996 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b255a-f914-440e-9577-ae72370bde74 None admin] RBAC: Adding query filter params (role.id=e1dc00443f96426e9edf6ea6ca8ae542, scope.project.id=191069caccd04233b012585e29cfa18b, user.id=1c0772205ae24713b8901666a87018b2) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:37.598519 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b255a-f914-440e-9577-ae72370bde74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:37.601159 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b255a-f914-440e-9577-ae72370bde74 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:37.612143 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 443/901] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:09:37 2022] GET /identity/v3/role_assignments?user.id=1c0772205ae24713b8901666a87018b2&scope.project.id=191069caccd04233b012585e29cfa18b&role.id=e1dc00443f96426e9edf6ea6ca8ae542 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:09:37.614336 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e8551-bf55-413f-bb6e-e89fcc7c7007 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:37.614821 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e8551-bf55-413f-bb6e-e89fcc7c7007 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:37.615304 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e8551-bf55-413f-bb6e-e89fcc7c7007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:37.636821 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f34e8551-bf55-413f-bb6e-e89fcc7c7007 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:37.638125 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f34e8551-bf55-413f-bb6e-e89fcc7c7007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:37.652812 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:37.684098 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f6a85-82d0-4a33-a186-ee5b01790af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:37.685725 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f6a85-82d0-4a33-a186-ee5b01790af6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:37.686630 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f6a85-82d0-4a33-a186-ee5b01790af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:37.687381 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f6a85-82d0-4a33-a186-ee5b01790af6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:37.689259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675f6a85-82d0-4a33-a186-ee5b01790af6 None admin] RBAC: Adding query filter params (interface=public, service_id=256571a90c4b43578f8e05105adab2ee, region_id=RegionOne) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:37.690087 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675f6a85-82d0-4a33-a186-ee5b01790af6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:37.693091 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675f6a85-82d0-4a33-a186-ee5b01790af6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:37.704936 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 459/902] 10.222.0.18 () {56 vars in 1251 bytes} [Mon Jul 25 19:09:37 2022] GET /identity/v3/endpoints?service_id=256571a90c4b43578f8e05105adab2ee&interface=public®ion_id=RegionOne => generated 194 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:37.711280 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f6a85-82d0-4a33-a186-ee5b01790af6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:37.727337 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 444/903] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:37 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:09:37.765445 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e8551-bf55-413f-bb6e-e89fcc7c7007 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:37.914323 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f6a85-82d0-4a33-a186-ee5b01790af6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:37.943592 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e8551-bf55-413f-bb6e-e89fcc7c7007 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:37.973185 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9198b6c-9ba2-4a6c-a63e-d6df814c5b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:37.973995 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9198b6c-9ba2-4a6c-a63e-d6df814c5b29 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:37.974335 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9198b6c-9ba2-4a6c-a63e-d6df814c5b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:37.974708 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9198b6c-9ba2-4a6c-a63e-d6df814c5b29 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:37.975764 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9198b6c-9ba2-4a6c-a63e-d6df814c5b29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:37.977013 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9198b6c-9ba2-4a6c-a63e-d6df814c5b29 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:37.985109 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 460/904] 10.222.0.18 () {56 vars in 1087 bytes} [Mon Jul 25 19:09:37 2022] GET /identity/v3/services => generated 1268 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:09:38.020205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f4263a-4850-42ab-819a-751b61de393b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:38.021157 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4263a-4850-42ab-819a-751b61de393b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:38.021481 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4263a-4850-42ab-819a-751b61de393b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:38.021816 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4263a-4850-42ab-819a-751b61de393b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:38.022812 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f4263a-4850-42ab-819a-751b61de393b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:38.024493 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f4263a-4850-42ab-819a-751b61de393b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:38.042962 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-30f4263a-4850-42ab-819a-751b61de393b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 19:09:38.042962 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:38.042962 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 19:09:38.042962 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 19:09:38.042962 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 19:09:38.042962 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 19:09:38.042962 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 19:09:38.042962 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:38.042962 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 19:09:38.042962 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:38.042962 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 19:09:38.042962 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:38.042962 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:38.042962 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:38.042962 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:38.042962 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:38.044903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:38.044903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:38.044903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:38.044903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:38.044903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:38.044903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:38.044903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:38.044903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 19:09:38.044903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 19:09:38.044903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 19:09:38.044903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 19:09:38.044903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:38.044903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:38.044903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:38.044903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:38.044903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:38.046551 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:38.046551 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:38.046551 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:38.046551 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:38.046551 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:38.046551 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:38.046551 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:38.046551 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:38.046551 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:38.046551 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:38.046551 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:38.046551 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 19:09:38.046551 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:38.046551 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 19:09:38.046551 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:38.046551 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 445/905] 10.222.0.18 () {56 vars in 1132 bytes} [Mon Jul 25 19:09:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:09:38.060742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9198b6c-9ba2-4a6c-a63e-d6df814c5b29 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:38.084251 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:38.226054 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9198b6c-9ba2-4a6c-a63e-d6df814c5b29 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:38.293828 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1187b276-5d9f-49f6-a150-5faa949c7514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:38.294965 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187b276-5d9f-49f6-a150-5faa949c7514 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:38.295428 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187b276-5d9f-49f6-a150-5faa949c7514 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:38.295918 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187b276-5d9f-49f6-a150-5faa949c7514 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:38.296737 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1187b276-5d9f-49f6-a150-5faa949c7514 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:38.297043 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1187b276-5d9f-49f6-a150-5faa949c7514 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:38.298474 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1187b276-5d9f-49f6-a150-5faa949c7514 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:38.304500 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 461/906] 10.222.0.18 () {56 vars in 1126 bytes} [Mon Jul 25 19:09:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:09:38.314684 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f4263a-4850-42ab-819a-751b61de393b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:38.338562 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:38.472601 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f4263a-4850-42ab-819a-751b61de393b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:38.517529 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1044c8bd-5d96-49cd-a9c3-c9aede9071e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:38.518234 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1044c8bd-5d96-49cd-a9c3-c9aede9071e6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:38.518521 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1044c8bd-5d96-49cd-a9c3-c9aede9071e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:38.518803 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1044c8bd-5d96-49cd-a9c3-c9aede9071e6 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:38.519795 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1044c8bd-5d96-49cd-a9c3-c9aede9071e6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:38.520179 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1044c8bd-5d96-49cd-a9c3-c9aede9071e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:38.521704 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1044c8bd-5d96-49cd-a9c3-c9aede9071e6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:38.528588 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 446/907] 10.222.0.18 () {56 vars in 1126 bytes} [Mon Jul 25 19:09:38 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:38.538949 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1187b276-5d9f-49f6-a150-5faa949c7514 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:38.582567 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112f5d1-36aa-45db-8a78-92d9890801b5 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:38.583007 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112f5d1-36aa-45db-8a78-92d9890801b5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:38.583239 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112f5d1-36aa-45db-8a78-92d9890801b5 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:38.585072 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 447/908] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:38.592288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad8500-533b-4634-a6ff-8eb09520c0fa None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:38.592708 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad8500-533b-4634-a6ff-8eb09520c0fa None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:38.592924 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad8500-533b-4634-a6ff-8eb09520c0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:38.622288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6cad8500-533b-4634-a6ff-8eb09520c0fa None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:38.624303 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6cad8500-533b-4634-a6ff-8eb09520c0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:38.657075 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:38.696000 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1187b276-5d9f-49f6-a150-5faa949c7514 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:38.731782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 448/909] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:38 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 19:09:38.744175 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb569c-ea27-46f9-8ec0-c5b54274ed0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:38.745927 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb569c-ea27-46f9-8ec0-c5b54274ed0e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:38.746442 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb569c-ea27-46f9-8ec0-c5b54274ed0e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:38.746928 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb569c-ea27-46f9-8ec0-c5b54274ed0e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:38.748489 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb569c-ea27-46f9-8ec0-c5b54274ed0e None admin] RBAC: Adding query filter params (interface=public, service_id=5e6e8e4149774158b63c91a0b5d00e7e, region_id=RegionOne) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:38.749127 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb569c-ea27-46f9-8ec0-c5b54274ed0e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:38.751802 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb569c-ea27-46f9-8ec0-c5b54274ed0e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:38.756700 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad8500-533b-4634-a6ff-8eb09520c0fa None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:38.761915 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:38.765285 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 462/910] 10.222.0.18 () {56 vars in 1251 bytes} [Mon Jul 25 19:09:38 2022] GET /identity/v3/endpoints?service_id=5e6e8e4149774158b63c91a0b5d00e7e&interface=public®ion_id=RegionOne => generated 194 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:38.772176 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb569c-ea27-46f9-8ec0-c5b54274ed0e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:38.894483 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad8500-533b-4634-a6ff-8eb09520c0fa None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:38.939323 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e9c55c-362f-4c41-865a-9dc169f8f740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:38.940388 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9c55c-362f-4c41-865a-9dc169f8f740 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:38.940894 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9c55c-362f-4c41-865a-9dc169f8f740 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:38.941356 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9c55c-362f-4c41-865a-9dc169f8f740 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:38.942459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e9c55c-362f-4c41-865a-9dc169f8f740 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:38.944307 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e9c55c-362f-4c41-865a-9dc169f8f740 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:38.973289 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 449/911] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Jul 25 19:09:38 2022] POST /identity/v3/services => generated 241 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:09:38.992785 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb569c-ea27-46f9-8ec0-c5b54274ed0e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:39.070419 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:39.079982 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1be00-2cef-4e93-b5f8-79caa5dd7d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:39.081178 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1be00-2cef-4e93-b5f8-79caa5dd7d8e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:39.081682 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1be00-2cef-4e93-b5f8-79caa5dd7d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:39.082246 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1be00-2cef-4e93-b5f8-79caa5dd7d8e None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:39.083621 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a1be00-2cef-4e93-b5f8-79caa5dd7d8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:39.086360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a1be00-2cef-4e93-b5f8-79caa5dd7d8e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:39.101042 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 463/912] 10.222.0.18 () {56 vars in 1087 bytes} [Mon Jul 25 19:09:38 2022] GET /identity/v3/services => generated 1497 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:09:40.620074 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb31ff-087d-48d2-b0cf-995eb50ab7ab None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:40.620917 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb31ff-087d-48d2-b0cf-995eb50ab7ab None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:40.620917 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb31ff-087d-48d2-b0cf-995eb50ab7ab None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:40.622789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 450/913] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:40.631184 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c832f0f2-050b-4f9e-8805-d6680c3a281a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:40.632170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c832f0f2-050b-4f9e-8805-d6680c3a281a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:40.632436 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c832f0f2-050b-4f9e-8805-d6680c3a281a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:40.654239 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c832f0f2-050b-4f9e-8805-d6680c3a281a None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:40.655623 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c832f0f2-050b-4f9e-8805-d6680c3a281a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:40.798154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 464/914] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 19:09:40.823002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb31ff-087d-48d2-b0cf-995eb50ab7ab None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:40.907897 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:40.933737 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39acfd-25e5-4bd7-9f08-739c64b6e542 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:40.934413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39acfd-25e5-4bd7-9f08-739c64b6e542 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:40.934955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da39acfd-25e5-4bd7-9f08-739c64b6e542 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:40.937333 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 465/915] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:40.943881 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3971ae3a-e2c6-41ba-b2ab-2791e9e6b0ed None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:40.944520 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3971ae3a-e2c6-41ba-b2ab-2791e9e6b0ed None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:40.945064 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3971ae3a-e2c6-41ba-b2ab-2791e9e6b0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:40.959196 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb31ff-087d-48d2-b0cf-995eb50ab7ab None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:40.968386 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3971ae3a-e2c6-41ba-b2ab-2791e9e6b0ed None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:40.970123 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3971ae3a-e2c6-41ba-b2ab-2791e9e6b0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:41.004199 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff1e37-599f-4c03-a1f0-0508a7171474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:41.005044 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff1e37-599f-4c03-a1f0-0508a7171474 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:41.005399 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff1e37-599f-4c03-a1f0-0508a7171474 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:41.005715 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff1e37-599f-4c03-a1f0-0508a7171474 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:41.006398 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ff1e37-599f-4c03-a1f0-0508a7171474 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:41.008301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ff1e37-599f-4c03-a1f0-0508a7171474 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:41.022831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-e4ff1e37-599f-4c03-a1f0-0508a7171474 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 19:09:41.022831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:41.022831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 19:09:41.022831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 19:09:41.022831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 19:09:41.022831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 19:09:41.022831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 19:09:41.022831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:41.022831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 19:09:41.022831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:41.022831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 19:09:41.022831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:41.022831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:41.022831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:41.022831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:41.022831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:41.022831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:41.023984 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:41.023984 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:41.023984 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:41.023984 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:41.023984 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:41.023984 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:41.023984 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 19:09:41.023984 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 19:09:41.023984 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 19:09:41.023984 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 19:09:41.023984 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:41.023984 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:41.023984 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:41.023984 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:41.023984 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:41.023984 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:41.024903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:41.024903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:41.024903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:41.024903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:41.024903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:41.024903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:41.024903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:41.024903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:41.024903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:41.024903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:41.024903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 19:09:41.024903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:41.024903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 19:09:41.024903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:41.024903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 451/916] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Jul 25 19:09:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:41.034934 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff1e37-599f-4c03-a1f0-0508a7171474 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:41.075893 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 466/917] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 19:09:41.092410 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3971ae3a-e2c6-41ba-b2ab-2791e9e6b0ed None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:41.172594 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:41.232465 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3971ae3a-e2c6-41ba-b2ab-2791e9e6b0ed None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:41.240953 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff1e37-599f-4c03-a1f0-0508a7171474 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:41.278193 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:41.288283 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a57246-7b71-4bee-9ebf-cb654ef688d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:41.289259 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a57246-7b71-4bee-9ebf-cb654ef688d5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:41.289514 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9664d36-a990-4c35-8b0a-b81c1bf4164c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:41.289744 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a57246-7b71-4bee-9ebf-cb654ef688d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:41.290179 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a57246-7b71-4bee-9ebf-cb654ef688d5 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:41.290919 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9664d36-a990-4c35-8b0a-b81c1bf4164c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:41.291337 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a57246-7b71-4bee-9ebf-cb654ef688d5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:41.291582 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9664d36-a990-4c35-8b0a-b81c1bf4164c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:41.292022 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9664d36-a990-4c35-8b0a-b81c1bf4164c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:41.292557 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a57246-7b71-4bee-9ebf-cb654ef688d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:41.294566 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a57246-7b71-4bee-9ebf-cb654ef688d5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:41.301343 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 452/918] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Jul 25 19:09:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:41.309414 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9664d36-a990-4c35-8b0a-b81c1bf4164c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:09:41.313822 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a57246-7b71-4bee-9ebf-cb654ef688d5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:41.318980 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-c9664d36-a990-4c35-8b0a-b81c1bf4164c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 19:09:41.318980 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:41.318980 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:41.318980 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:41.318980 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:41.318980 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:41.318980 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:41.318980 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:41.318980 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:41.318980 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:41.318980 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:41.318980 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:41.318980 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:09:41.318980 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:09:41.318980 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:09:41.318980 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:09:41.320896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:09:41.320896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:09:41.320896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:09:41.320896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:09:41.320896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:41.320896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:41.320896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:41.320896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:41.320896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:41.320896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:41.320896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:41.320896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:41.320896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:41.320896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:41.320896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:41.320896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:41.322660 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:41.322660 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:41.322660 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:41.322660 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:41.322660 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:09:41.322660 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:09:41.322660 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:09:41.322660 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:09:41.322660 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:09:41.322660 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:09:41.322660 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 19:09:41.322660 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:41.326490 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 467/919] 10.222.0.18 () {56 vars in 1120 bytes} [Mon Jul 25 19:09:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:41.327581 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9664d36-a990-4c35-8b0a-b81c1bf4164c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:41.464766 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9664d36-a990-4c35-8b0a-b81c1bf4164c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:41.488766 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:41.512995 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ffd2b-4c8d-46f1-8de7-a701ec10a6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:41.513833 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ffd2b-4c8d-46f1-8de7-a701ec10a6eb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:41.514204 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ffd2b-4c8d-46f1-8de7-a701ec10a6eb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:41.514498 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ffd2b-4c8d-46f1-8de7-a701ec10a6eb None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:41.515336 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3ffd2b-4c8d-46f1-8de7-a701ec10a6eb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:41.515788 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3ffd2b-4c8d-46f1-8de7-a701ec10a6eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:41.517466 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3ffd2b-4c8d-46f1-8de7-a701ec10a6eb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:41.532001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 468/920] 10.222.0.18 () {56 vars in 1115 bytes} [Mon Jul 25 19:09:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:09:41.536512 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ffd2b-4c8d-46f1-8de7-a701ec10a6eb None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:41.553090 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a57246-7b71-4bee-9ebf-cb654ef688d5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:41.624581 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c4ad6d-d017-414c-a3d8-1fd861d2f05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:41.625892 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4ad6d-d017-414c-a3d8-1fd861d2f05f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:41.626376 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4ad6d-d017-414c-a3d8-1fd861d2f05f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:41.626873 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4ad6d-d017-414c-a3d8-1fd861d2f05f None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:41.628516 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c4ad6d-d017-414c-a3d8-1fd861d2f05f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:41.628809 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c4ad6d-d017-414c-a3d8-1fd861d2f05f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:41.631464 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c4ad6d-d017-414c-a3d8-1fd861d2f05f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:41.646801 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 453/921] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Jul 25 19:09:41 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:41.646801 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2bf59-908f-42cf-9a87-08cd8ae0b647 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:41.646801 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2bf59-908f-42cf-9a87-08cd8ae0b647 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:41.646801 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2bf59-908f-42cf-9a87-08cd8ae0b647 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:41.648852 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 454/922] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:41.649934 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2bf59-908f-42cf-9a87-08cd8ae0b647 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:41.728087 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ffd2b-4c8d-46f1-8de7-a701ec10a6eb None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:41.775590 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:41.792257 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3719d0-6697-44c4-93fe-cc9640864cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:41.793282 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3719d0-6697-44c4-93fe-cc9640864cee None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:41.793664 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3719d0-6697-44c4-93fe-cc9640864cee None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:41.793947 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3719d0-6697-44c4-93fe-cc9640864cee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:41.809149 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3719d0-6697-44c4-93fe-cc9640864cee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:41.810796 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3719d0-6697-44c4-93fe-cc9640864cee None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:41.834450 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-8e3719d0-6697-44c4-93fe-cc9640864cee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:41.834450 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:41.834450 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:41.834450 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:41.834450 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:41.834450 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:41.834450 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:41.834450 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:41.834450 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:41.834450 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:41.834450 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:41.834450 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:41.834450 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:09:41.834450 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:09:41.834450 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:09:41.836398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:09:41.836398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:41.836398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:41.836398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:41.836398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:41.836398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:41.836398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:41.836398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:41.836398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:41.836398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:41.836398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:41.836398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:41.836398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:41.836398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:41.836398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:41.836398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:41.836398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:41.838292 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:09:41.838292 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:09:41.838292 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:41.838292 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:41.838292 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 469/923] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:09:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:41.838844 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5d2a7-b675-435d-9c08-2bc472707921 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:41.839334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5d2a7-b675-435d-9c08-2bc472707921 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:41.839922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5d2a7-b675-435d-9c08-2bc472707921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:41.866595 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-56f5d2a7-b675-435d-9c08-2bc472707921 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:41.868276 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-56f5d2a7-b675-435d-9c08-2bc472707921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:41.883472 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2bf59-908f-42cf-9a87-08cd8ae0b647 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:41.907127 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:41.935877 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a101ea5-93a4-4f26-9ea9-4a67d898590d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:41.936579 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a101ea5-93a4-4f26-9ea9-4a67d898590d None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:41.936872 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a101ea5-93a4-4f26-9ea9-4a67d898590d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:41.937160 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a101ea5-93a4-4f26-9ea9-4a67d898590d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:41.937947 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a101ea5-93a4-4f26-9ea9-4a67d898590d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:41.939396 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a101ea5-93a4-4f26-9ea9-4a67d898590d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:41.978430 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 455/924] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:09:41 2022] POST /identity/v3/endpoints => generated 346 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:09:41.978430 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a101ea5-93a4-4f26-9ea9-4a67d898590d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:41.988816 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 470/925] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:09:41.991567 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44edb6a5-c7c8-4cb1-a634-c73bff09d7bf None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:41.992046 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44edb6a5-c7c8-4cb1-a634-c73bff09d7bf None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:41.992514 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44edb6a5-c7c8-4cb1-a634-c73bff09d7bf None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:41.997160 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 471/926] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:42.009278 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6778a-b351-43c1-bb6d-2571e9096157 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:42.009278 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6778a-b351-43c1-bb6d-2571e9096157 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:42.009278 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6778a-b351-43c1-bb6d-2571e9096157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:42.035874 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5cc6778a-b351-43c1-bb6d-2571e9096157 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:42.037526 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5cc6778a-b351-43c1-bb6d-2571e9096157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:42.147801 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 472/927] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 19:09:42.162234 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6778a-b351-43c1-bb6d-2571e9096157 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:42.163963 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a101ea5-93a4-4f26-9ea9-4a67d898590d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:42.189068 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:42.223280 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed98b11-7bd7-4d4f-a802-5377a77a0817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:42.224146 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed98b11-7bd7-4d4f-a802-5377a77a0817 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:42.224485 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed98b11-7bd7-4d4f-a802-5377a77a0817 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:42.224648 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed98b11-7bd7-4d4f-a802-5377a77a0817 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:42.225375 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed98b11-7bd7-4d4f-a802-5377a77a0817 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:42.225684 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed98b11-7bd7-4d4f-a802-5377a77a0817 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:42.227343 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed98b11-7bd7-4d4f-a802-5377a77a0817 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:42.249572 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 456/928] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:42.250417 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed98b11-7bd7-4d4f-a802-5377a77a0817 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:42.311135 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:42.334205 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6778a-b351-43c1-bb6d-2571e9096157 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:42.384705 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ba347-d965-4e2d-b3bc-b2c2088cceb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:42.386349 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ba347-d965-4e2d-b3bc-b2c2088cceb8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:42.386631 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ba347-d965-4e2d-b3bc-b2c2088cceb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:42.386907 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ba347-d965-4e2d-b3bc-b2c2088cceb8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:42.387686 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ba347-d965-4e2d-b3bc-b2c2088cceb8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:42.389134 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ba347-d965-4e2d-b3bc-b2c2088cceb8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:42.405649 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b29ba347-d965-4e2d-b3bc-b2c2088cceb8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 19:09:42.405649 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:42.405649 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 19:09:42.405649 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 19:09:42.405649 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 19:09:42.405649 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 19:09:42.405649 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 19:09:42.405649 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:42.405649 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 19:09:42.405649 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:42.405649 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 19:09:42.405649 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:42.405649 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:42.405649 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:42.405649 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:42.405649 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:42.405649 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:42.408135 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:42.408135 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:42.408135 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:42.408135 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:42.408135 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:42.408135 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:42.408135 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 19:09:42.408135 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 19:09:42.408135 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 19:09:42.408135 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 19:09:42.408135 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:42.408135 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:42.408135 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:42.408135 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:42.408135 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:42.408135 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:42.410485 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:42.410485 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:42.410485 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:42.410485 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:42.410485 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:42.410485 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:42.410485 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:42.410485 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:42.410485 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:42.410485 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:42.410485 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 19:09:42.410485 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:42.410485 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 19:09:42.410485 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:42.410485 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 473/929] 10.222.0.18 () {56 vars in 1126 bytes} [Mon Jul 25 19:09:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:42.412862 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ba347-d965-4e2d-b3bc-b2c2088cceb8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:42.469053 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed98b11-7bd7-4d4f-a802-5377a77a0817 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:42.546000 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98bf74-dd29-4b16-a7c2-7b28c080abb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:42.547360 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98bf74-dd29-4b16-a7c2-7b28c080abb0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:42.547854 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98bf74-dd29-4b16-a7c2-7b28c080abb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:42.548362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98bf74-dd29-4b16-a7c2-7b28c080abb0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:42.549598 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98bf74-dd29-4b16-a7c2-7b28c080abb0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:42.552354 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98bf74-dd29-4b16-a7c2-7b28c080abb0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:42.578326 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-ac98bf74-dd29-4b16-a7c2-7b28c080abb0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 19:09:42.578326 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:42.578326 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 19:09:42.578326 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 19:09:42.578326 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 19:09:42.578326 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 19:09:42.578326 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 19:09:42.578326 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:42.578326 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 19:09:42.578326 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:42.578326 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 19:09:42.578326 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:42.578326 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:42.578326 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:42.578326 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:42.578326 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:42.579377 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:42.579377 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:42.579377 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:42.579377 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:42.579377 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:42.579377 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:42.579377 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:42.579377 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 19:09:42.579377 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 19:09:42.579377 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 19:09:42.579377 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 19:09:42.579377 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:42.579377 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:42.579377 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:42.579377 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:42.579377 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:42.580287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:42.580287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:42.580287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:42.580287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:42.580287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:42.580287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:42.580287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:42.580287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:42.580287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:42.580287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:42.580287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:42.580287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 19:09:42.580287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:42.580287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 19:09:42.580287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:42.581097 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 457/930] 10.222.0.18 () {56 vars in 1132 bytes} [Mon Jul 25 19:09:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:42.582183 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98bf74-dd29-4b16-a7c2-7b28c080abb0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:42.615853 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ba347-d965-4e2d-b3bc-b2c2088cceb8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:42.638076 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:42.684726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a33c1ce-a368-4789-96b1-a400e19be415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:42.686322 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33c1ce-a368-4789-96b1-a400e19be415 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:42.686752 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33c1ce-a368-4789-96b1-a400e19be415 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:42.687342 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33c1ce-a368-4789-96b1-a400e19be415 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:42.702699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a33c1ce-a368-4789-96b1-a400e19be415 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:42.705486 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a33c1ce-a368-4789-96b1-a400e19be415 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:42.728208 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-9a33c1ce-a368-4789-96b1-a400e19be415 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 19:09:42.728208 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:42.728208 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:42.728208 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:42.728208 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:42.728208 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:42.728208 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:42.728208 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:42.728208 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:42.728208 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:42.728208 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:42.728208 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:42.728208 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:09:42.728208 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:09:42.728208 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:09:42.728208 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:09:42.730542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:42.730542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:42.730542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:09:42.730542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:09:42.730542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:09:42.730542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:09:42.730542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:42.730542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:42.730542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:42.730542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:42.730542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:42.730542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:42.730542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:42.730542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:42.730542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:42.730542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:42.730542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:42.730542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:42.731904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:42.731904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:42.731904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:09:42.731904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:09:42.731904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:09:42.731904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:09:42.731904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:09:42.731904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:09:42.731904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 19:09:42.731904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:42.735718 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 474/931] 10.222.0.18 () {56 vars in 1117 bytes} [Mon Jul 25 19:09:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:42.736297 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a33c1ce-a368-4789-96b1-a400e19be415 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:42.758525 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:42.771529 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98bf74-dd29-4b16-a7c2-7b28c080abb0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:42.846414 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd80f4e-da98-4f8c-945d-ba9953416088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:42.847536 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd80f4e-da98-4f8c-945d-ba9953416088 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:42.848006 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd80f4e-da98-4f8c-945d-ba9953416088 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:42.848461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd80f4e-da98-4f8c-945d-ba9953416088 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:42.849736 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd80f4e-da98-4f8c-945d-ba9953416088 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:42.850239 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd80f4e-da98-4f8c-945d-ba9953416088 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:42.852522 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd80f4e-da98-4f8c-945d-ba9953416088 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:42.862330 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 458/932] 10.222.0.18 () {56 vars in 1122 bytes} [Mon Jul 25 19:09:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:42.863863 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd80f4e-da98-4f8c-945d-ba9953416088 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:42.923916 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a33c1ce-a368-4789-96b1-a400e19be415 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:43.003082 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021118ec-dcea-435d-a2bc-a9632a0df4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:43.004330 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021118ec-dcea-435d-a2bc-a9632a0df4cb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:43.004832 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021118ec-dcea-435d-a2bc-a9632a0df4cb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:43.005338 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021118ec-dcea-435d-a2bc-a9632a0df4cb None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:43.006713 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021118ec-dcea-435d-a2bc-a9632a0df4cb None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:43.007270 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021118ec-dcea-435d-a2bc-a9632a0df4cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:43.010562 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021118ec-dcea-435d-a2bc-a9632a0df4cb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:43.025324 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 475/933] 10.222.0.18 () {56 vars in 1126 bytes} [Mon Jul 25 19:09:42 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:43.025324 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021118ec-dcea-435d-a2bc-a9632a0df4cb None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:43.045602 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:43.049403 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd80f4e-da98-4f8c-945d-ba9953416088 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:43.112697 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a97658-9ccb-4d0e-8d75-6f8a9235a14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:43.113790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a97658-9ccb-4d0e-8d75-6f8a9235a14c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:43.114300 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a97658-9ccb-4d0e-8d75-6f8a9235a14c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:43.114792 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a97658-9ccb-4d0e-8d75-6f8a9235a14c None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:43.116544 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a97658-9ccb-4d0e-8d75-6f8a9235a14c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:43.117101 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a97658-9ccb-4d0e-8d75-6f8a9235a14c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:43.120537 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a97658-9ccb-4d0e-8d75-6f8a9235a14c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:43.150727 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:43.156068 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 459/934] 10.222.0.18 () {56 vars in 1149 bytes} [Mon Jul 25 19:09:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:43.157692 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a97658-9ccb-4d0e-8d75-6f8a9235a14c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:43.207814 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021118ec-dcea-435d-a2bc-a9632a0df4cb None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:43.273013 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35331537-d81e-457d-b115-a086d5e1bbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:43.273720 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35331537-d81e-457d-b115-a086d5e1bbf1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:43.274139 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35331537-d81e-457d-b115-a086d5e1bbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:43.274463 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35331537-d81e-457d-b115-a086d5e1bbf1 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:43.275313 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35331537-d81e-457d-b115-a086d5e1bbf1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:43.275815 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35331537-d81e-457d-b115-a086d5e1bbf1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:43.277486 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35331537-d81e-457d-b115-a086d5e1bbf1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:43.293173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 476/935] 10.222.0.18 () {56 vars in 1122 bytes} [Mon Jul 25 19:09:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:43.293173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35331537-d81e-457d-b115-a086d5e1bbf1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:43.321906 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a97658-9ccb-4d0e-8d75-6f8a9235a14c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:43.392876 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b1715-d4c0-472d-9ac3-b99393691a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:43.393875 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b1715-d4c0-472d-9ac3-b99393691a41 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:43.394343 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b1715-d4c0-472d-9ac3-b99393691a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:43.394793 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b1715-d4c0-472d-9ac3-b99393691a41 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:43.396067 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b1715-d4c0-472d-9ac3-b99393691a41 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:43.396575 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b1715-d4c0-472d-9ac3-b99393691a41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:43.398827 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b1715-d4c0-472d-9ac3-b99393691a41 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:43.415554 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 460/936] 10.222.0.18 () {56 vars in 1126 bytes} [Mon Jul 25 19:09:43 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:43.415554 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b1715-d4c0-472d-9ac3-b99393691a41 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:43.467839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35331537-d81e-457d-b115-a086d5e1bbf1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:43.469340 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:43.549284 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8157a4-ad7a-49f9-80e9-6eff4f62bb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:43.550346 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8157a4-ad7a-49f9-80e9-6eff4f62bb66 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:43.550912 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8157a4-ad7a-49f9-80e9-6eff4f62bb66 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:43.551387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8157a4-ad7a-49f9-80e9-6eff4f62bb66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:43.563765 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:43.571237 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8157a4-ad7a-49f9-80e9-6eff4f62bb66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:43.573291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8157a4-ad7a-49f9-80e9-6eff4f62bb66 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:43.601274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-7d8157a4-ad7a-49f9-80e9-6eff4f62bb66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:43.601274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:43.601274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:43.601274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:43.601274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:43.601274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:43.601274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:43.601274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:43.601274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:43.601274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:43.601274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:43.601274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:43.601274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:09:43.601274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:09:43.601274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:09:43.602312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:09:43.602312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:43.602312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:43.602312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:43.602312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:43.602312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:43.602312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:43.602312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:43.602312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:43.602312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:43.602312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:43.602312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:43.602312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:43.602312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:43.602312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:43.602312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:43.602312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:43.604420 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:09:43.604420 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:09:43.604420 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:43.604420 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:43.604420 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 477/937] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:09:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:43.605005 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8157a4-ad7a-49f9-80e9-6eff4f62bb66 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:43.623194 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b1715-d4c0-472d-9ac3-b99393691a41 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:43.671620 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628360c7-8f3b-47f3-8fd9-9a0579c8d6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:43.672249 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628360c7-8f3b-47f3-8fd9-9a0579c8d6c3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:43.672511 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628360c7-8f3b-47f3-8fd9-9a0579c8d6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:43.672790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628360c7-8f3b-47f3-8fd9-9a0579c8d6c3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:43.673627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628360c7-8f3b-47f3-8fd9-9a0579c8d6c3 None admin] RBAC: Adding query filter params (interface=public, service_id=ebda878503be465e98903768e3143079, region_id=RegionOne) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:43.673947 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628360c7-8f3b-47f3-8fd9-9a0579c8d6c3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:43.675502 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628360c7-8f3b-47f3-8fd9-9a0579c8d6c3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:43.693184 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 461/938] 10.222.0.18 () {56 vars in 1251 bytes} [Mon Jul 25 19:09:43 2022] GET /identity/v3/endpoints?service_id=ebda878503be465e98903768e3143079&interface=public®ion_id=RegionOne => generated 194 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:43.693942 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628360c7-8f3b-47f3-8fd9-9a0579c8d6c3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:43.814877 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8157a4-ad7a-49f9-80e9-6eff4f62bb66 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:43.845016 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:43.866412 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d78ab0e-7a75-4153-9b26-1a9e8524b583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:43.867483 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78ab0e-7a75-4153-9b26-1a9e8524b583 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:43.867676 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78ab0e-7a75-4153-9b26-1a9e8524b583 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:43.867967 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78ab0e-7a75-4153-9b26-1a9e8524b583 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:43.868743 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d78ab0e-7a75-4153-9b26-1a9e8524b583 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:43.869777 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628360c7-8f3b-47f3-8fd9-9a0579c8d6c3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:43.870699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d78ab0e-7a75-4153-9b26-1a9e8524b583 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:43.899869 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 478/939] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:09:43 2022] POST /identity/v3/endpoints => generated 347 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:09:43.909756 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d78ab0e-7a75-4153-9b26-1a9e8524b583 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:43.932677 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f945cf-0c6a-432e-a858-da815f9be9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:43.933388 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f945cf-0c6a-432e-a858-da815f9be9b2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:43.934027 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f945cf-0c6a-432e-a858-da815f9be9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:43.934375 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f945cf-0c6a-432e-a858-da815f9be9b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:43.935638 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f945cf-0c6a-432e-a858-da815f9be9b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:43.935638 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f945cf-0c6a-432e-a858-da815f9be9b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:43.937472 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f945cf-0c6a-432e-a858-da815f9be9b2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:43.963546 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 462/940] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:09:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:43.969774 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f945cf-0c6a-432e-a858-da815f9be9b2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:44.018623 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:44.100476 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d78ab0e-7a75-4153-9b26-1a9e8524b583 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:44.138016 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f945cf-0c6a-432e-a858-da815f9be9b2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:44.154050 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3088797-7de5-48fb-b3d9-0f93405242c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:44.154798 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3088797-7de5-48fb-b3d9-0f93405242c6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:44.155112 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3088797-7de5-48fb-b3d9-0f93405242c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:44.155390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3088797-7de5-48fb-b3d9-0f93405242c6 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:44.156181 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3088797-7de5-48fb-b3d9-0f93405242c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:44.157738 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3088797-7de5-48fb-b3d9-0f93405242c6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:44.167575 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 479/941] 10.222.0.18 () {56 vars in 1087 bytes} [Mon Jul 25 19:09:43 2022] GET /identity/v3/services => generated 1497 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:09:44.214837 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da60c0-9108-42a3-9fa0-6417d7a63261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:44.216603 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da60c0-9108-42a3-9fa0-6417d7a63261 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:44.217098 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da60c0-9108-42a3-9fa0-6417d7a63261 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:44.217534 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da60c0-9108-42a3-9fa0-6417d7a63261 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:44.228452 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1da60c0-9108-42a3-9fa0-6417d7a63261 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:44.230892 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1da60c0-9108-42a3-9fa0-6417d7a63261 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:44.250308 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-a1da60c0-9108-42a3-9fa0-6417d7a63261 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:09:44.250308 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:44.250308 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:44.250308 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:44.250308 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:44.250308 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:44.250308 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:44.250308 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:44.250308 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:44.250308 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:44.250308 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:44.250308 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:44.250308 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:09:44.250308 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:09:44.250308 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:09:44.251972 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:09:44.251972 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:44.251972 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:44.251972 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:44.251972 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:44.251972 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:44.251972 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:44.251972 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:44.251972 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:44.251972 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:44.251972 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:44.251972 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:44.251972 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:44.251972 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:44.251972 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:44.251972 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:44.251972 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:44.253090 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:09:44.253090 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:09:44.253090 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:09:44.253090 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:09:44.253090 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:09:44.253090 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:09:44.253090 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:09:44.253090 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:44.257750 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 463/942] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:09:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:09:44.261954 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3088797-7de5-48fb-b3d9-0f93405242c6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:44.421949 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3088797-7de5-48fb-b3d9-0f93405242c6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:44.437431 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:44.474097 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be776789-72b4-4940-b35f-1257b36252ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:44.474826 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be776789-72b4-4940-b35f-1257b36252ed None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:44.475205 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be776789-72b4-4940-b35f-1257b36252ed None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:44.475471 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be776789-72b4-4940-b35f-1257b36252ed None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:44.476415 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be776789-72b4-4940-b35f-1257b36252ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:44.476804 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be776789-72b4-4940-b35f-1257b36252ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:44.478337 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be776789-72b4-4940-b35f-1257b36252ed None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:44.497090 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 480/943] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Jul 25 19:09:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:09:44.503225 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da60c0-9108-42a3-9fa0-6417d7a63261 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:44.508791 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:44.674452 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da60c0-9108-42a3-9fa0-6417d7a63261 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:44.736995 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44469c8f-812d-4dd3-8991-196076843cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:44.738540 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44469c8f-812d-4dd3-8991-196076843cd3 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:44.739022 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44469c8f-812d-4dd3-8991-196076843cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:44.739347 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44469c8f-812d-4dd3-8991-196076843cd3 None admin] PATH_INFO: `/v3/projects/191069caccd04233b012585e29cfa18b/users/1c0772205ae24713b8901666a87018b2/roles/e1dc00443f96426e9edf6ea6ca8ae542` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:44.742520 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44469c8f-812d-4dd3-8991-196076843cd3 None admin] RBAC: Authorizing `identity:create_grant(project_id=191069caccd04233b012585e29cfa18b, user_id=1c0772205ae24713b8901666a87018b2, role_id=e1dc00443f96426e9edf6ea6ca8ae542)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:44.744702 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44469c8f-812d-4dd3-8991-196076843cd3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:44.764558 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 464/944] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Jul 25 19:09:44 2022] PUT /identity/v3/projects/191069caccd04233b012585e29cfa18b/users/1c0772205ae24713b8901666a87018b2/roles/e1dc00443f96426e9edf6ea6ca8ae542 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:09:45.369273 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17214a-ac96-4425-9cc8-669e31908ff9 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:45.370090 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17214a-ac96-4425-9cc8-669e31908ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:45.370227 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17214a-ac96-4425-9cc8-669e31908ff9 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:45.372922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 481/945] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:45.382120 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80814e12-324f-4a9b-bf03-edaf2c723c66 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:45.382627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80814e12-324f-4a9b-bf03-edaf2c723c66 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:45.383136 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80814e12-324f-4a9b-bf03-edaf2c723c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:45.419133 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-80814e12-324f-4a9b-bf03-edaf2c723c66 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:45.421372 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-80814e12-324f-4a9b-bf03-edaf2c723c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:45.505466 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:45.529102 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 465/946] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:09:45.570988 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a17214a-ac96-4425-9cc8-669e31908ff9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:45.803897 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a17214a-ac96-4425-9cc8-669e31908ff9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:45.876106 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed69dc-316d-4dc7-8428-c10da9322ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:45.877218 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed69dc-316d-4dc7-8428-c10da9322ad9 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:45.877712 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed69dc-316d-4dc7-8428-c10da9322ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:45.878187 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed69dc-316d-4dc7-8428-c10da9322ad9 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:45.879537 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed69dc-316d-4dc7-8428-c10da9322ad9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:45.882300 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed69dc-316d-4dc7-8428-c10da9322ad9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:45.891551 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: WARNING keystone.server.flask.common [None req-12ed69dc-316d-4dc7-8428-c10da9322ad9 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 25 19:09:45.924694 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.sql.core [None req-12ed69dc-316d-4dc7-8428-c10da9322ad9 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 25 19:09:45.924694 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 25 19:09:45.924694 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [parameters: {'id': 'ebb4ec7236fd4fa89c708a0b0b7b1f67', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 25 19:09:45.924694 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80237) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 25 19:09:45.942836 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-12ed69dc-316d-4dc7-8428-c10da9322ad9 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ebb4ec7236fd4fa89c708a0b0b7b1f67.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ebb4ec7236fd4fa89c708a0b0b7b1f67. Jul 25 19:09:45.942836 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:45.942836 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 19:09:45.942836 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 19:09:45.942836 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 19:09:45.942836 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 19:09:45.942836 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 19:09:45.942836 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 19:09:45.942836 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 19:09:45.942836 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application conn.query(q) Jul 25 19:09:45.942836 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 19:09:45.942836 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 19:09:45.942836 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 19:09:45.944383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result.read() Jul 25 19:09:45.944383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 19:09:45.944383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 19:09:45.944383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 19:09:45.944383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 19:09:45.944383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 19:09:45.944383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 19:09:45.944383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 19:09:45.944383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 19:09:45.944383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 25 19:09:45.944383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:45.944383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 25 19:09:45.944383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:45.944383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:45.944383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 25 19:09:45.944383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 25 19:09:45.944383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 25 19:09:45.944383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 25 19:09:45.946702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 19:09:45.946702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application next(self.gen) Jul 25 19:09:45.946702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 25 19:09:45.946702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application yield resource Jul 25 19:09:45.946702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 19:09:45.946702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application next(self.gen) Jul 25 19:09:45.946702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 25 19:09:45.946702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.session.rollback() Jul 25 19:09:45.946702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 25 19:09:45.946702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.force_reraise() Jul 25 19:09:45.946702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 25 19:09:45.946702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise self.value Jul 25 19:09:45.946702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 25 19:09:45.946702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 25 19:09:45.946702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 25 19:09:45.946702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application session.commit() Jul 25 19:09:45.946702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 25 19:09:45.951344 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 25 19:09:45.951344 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 25 19:09:45.951344 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._prepare_impl() Jul 25 19:09:45.951344 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 25 19:09:45.951344 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.session.flush() Jul 25 19:09:45.951344 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 25 19:09:45.951344 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._flush(objects) Jul 25 19:09:45.951344 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 25 19:09:45.951344 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 25 19:09:45.951344 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 25 19:09:45.951344 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application compat.raise_( Jul 25 19:09:45.951344 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 19:09:45.951344 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception Jul 25 19:09:45.951344 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 25 19:09:45.951344 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application flush_context.execute() Jul 25 19:09:45.951344 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 25 19:09:45.951344 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rec.execute(self) Jul 25 19:09:45.960437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 25 19:09:45.960437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 25 19:09:45.960437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 25 19:09:45.960437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 25 19:09:45.960437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 25 19:09:45.960437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 25 19:09:45.960437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 25 19:09:45.960437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 25 19:09:45.960437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 25 19:09:45.960437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 25 19:09:45.960437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 25 19:09:45.960437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 25 19:09:45.960437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 25 19:09:45.960437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 25 19:09:45.960437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 25 19:09:45.964346 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 25 19:09:45.964346 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 19:09:45.964346 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception Jul 25 19:09:45.964346 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 19:09:45.964346 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 19:09:45.964346 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 19:09:45.964346 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 19:09:45.964346 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 19:09:45.964346 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 19:09:45.964346 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 19:09:45.964346 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application conn.query(q) Jul 25 19:09:45.964346 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 19:09:45.964346 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 19:09:45.964346 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 19:09:45.964346 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result.read() Jul 25 19:09:45.964346 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 19:09:45.964346 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 19:09:45.968017 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 19:09:45.968017 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 19:09:45.968017 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 19:09:45.968017 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 19:09:45.968017 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 19:09:45.968017 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 19:09:45.968017 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 25 19:09:45.968017 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 25 19:09:45.968017 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [parameters: {'id': 'ebb4ec7236fd4fa89c708a0b0b7b1f67', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 25 19:09:45.968017 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 25 19:09:45.968017 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:45.968017 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 19:09:45.968017 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:45.968017 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:45.968017 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 25 19:09:45.976547 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 25 19:09:45.976547 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 25 19:09:45.976547 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 25 19:09:45.976547 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 25 19:09:45.976547 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:45.976547 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 19:09:45.976547 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:45.976547 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:45.976547 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:45.976547 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:45.976547 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:45.976547 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:45.976547 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:45.976547 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:45.976547 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:45.976547 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:45.976547 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:45.980702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:45.980702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 25 19:09:45.980702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 25 19:09:45.980702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:45.980702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:45.980702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 25 19:09:45.980702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 25 19:09:45.980702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ebb4ec7236fd4fa89c708a0b0b7b1f67. Jul 25 19:09:45.980702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:45.980702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 482/947] 10.222.0.18 () {60 vars in 1140 bytes} [Mon Jul 25 19:09:45 2022] POST /identity/v3/projects => generated 262 bytes in 376 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:45.987918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80814e12-324f-4a9b-bf03-edaf2c723c66 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:46.182608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80814e12-324f-4a9b-bf03-edaf2c723c66 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:46.254993 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe38af-d407-49c2-96d5-c0e1c8f60e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:46.257102 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe38af-d407-49c2-96d5-c0e1c8f60e45 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:46.258002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe38af-d407-49c2-96d5-c0e1c8f60e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:46.258791 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe38af-d407-49c2-96d5-c0e1c8f60e45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:46.272348 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffe38af-d407-49c2-96d5-c0e1c8f60e45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:46.276198 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffe38af-d407-49c2-96d5-c0e1c8f60e45 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:46.304537 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-8ffe38af-d407-49c2-96d5-c0e1c8f60e45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:09:46.304537 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:46.304537 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:46.304537 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:46.304537 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:46.304537 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:46.304537 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:46.304537 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:46.304537 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:46.304537 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:46.304537 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:46.304537 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:46.304537 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:09:46.304537 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:09:46.304537 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:09:46.306233 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:09:46.306233 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:46.306233 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:46.306233 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:46.306233 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:46.306233 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:46.306233 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:46.306233 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:46.306233 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:46.306233 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:46.306233 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:46.306233 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:46.306233 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:46.306233 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:46.306233 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:46.306233 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:46.306233 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:46.308648 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:09:46.308648 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:09:46.308648 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:09:46.308648 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:09:46.308648 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:09:46.308648 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:09:46.308648 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:09:46.308648 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:46.313004 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 466/948] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:09:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:46.323662 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed69dc-316d-4dc7-8428-c10da9322ad9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:46.385144 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:46.481777 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070eac7a-6321-4f8a-9e06-75000fb21a00 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:46.482055 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070eac7a-6321-4f8a-9e06-75000fb21a00 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:46.482387 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070eac7a-6321-4f8a-9e06-75000fb21a00 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:46.485360 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 467/949] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:46.490285 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be29b7-f4bc-45ed-a6d3-065bf6de847f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:46.490709 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be29b7-f4bc-45ed-a6d3-065bf6de847f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:46.490902 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be29b7-f4bc-45ed-a6d3-065bf6de847f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:46.512603 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed69dc-316d-4dc7-8428-c10da9322ad9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:46.513868 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-93be29b7-f4bc-45ed-a6d3-065bf6de847f None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:46.515391 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-93be29b7-f4bc-45ed-a6d3-065bf6de847f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:46.560447 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1be67-d569-4c73-92cc-168d5d3a3fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:46.561140 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1be67-d569-4c73-92cc-168d5d3a3fdd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:46.561432 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1be67-d569-4c73-92cc-168d5d3a3fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:46.561708 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1be67-d569-4c73-92cc-168d5d3a3fdd None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:46.562694 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b1be67-d569-4c73-92cc-168d5d3a3fdd None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:46.563009 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b1be67-d569-4c73-92cc-168d5d3a3fdd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:46.565607 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b1be67-d569-4c73-92cc-168d5d3a3fdd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:46.596865 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 483/950] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Jul 25 19:09:46 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:46.629941 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:46.679893 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 468/951] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 19:09:46.704536 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1be67-d569-4c73-92cc-168d5d3a3fdd None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:46.824246 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:46.843118 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1be67-d569-4c73-92cc-168d5d3a3fdd None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:46.891226 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb563ca-ac3f-4a47-b109-ce7f12ef521d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:46.892632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb563ca-ac3f-4a47-b109-ce7f12ef521d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:46.893106 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb563ca-ac3f-4a47-b109-ce7f12ef521d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:46.893565 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb563ca-ac3f-4a47-b109-ce7f12ef521d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:46.894425 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb563ca-ac3f-4a47-b109-ce7f12ef521d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:46.896142 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb563ca-ac3f-4a47-b109-ce7f12ef521d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:46.910828 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-acb563ca-ac3f-4a47-b109-ce7f12ef521d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 19:09:46.910828 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:46.910828 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 19:09:46.910828 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 19:09:46.910828 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 19:09:46.910828 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 19:09:46.910828 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 19:09:46.910828 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:46.910828 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 19:09:46.910828 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:46.910828 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 19:09:46.910828 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:46.910828 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:46.910828 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:46.910828 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:46.910828 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:46.910828 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:46.912437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:46.912437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:46.912437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:46.912437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:46.912437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:46.912437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:46.912437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 19:09:46.912437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 19:09:46.912437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 19:09:46.912437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 19:09:46.912437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:46.912437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:46.912437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:46.912437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:46.912437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:46.912437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:46.913899 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:46.913899 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:46.913899 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:46.913899 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:46.913899 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:46.913899 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:46.913899 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:46.913899 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:46.913899 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:46.913899 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:46.913899 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 19:09:46.913899 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:46.913899 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 19:09:46.913899 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:46.922663 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 484/952] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:09:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:46.925148 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be29b7-f4bc-45ed-a6d3-065bf6de847f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:46.962568 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162aea3-5c00-4065-807c-57cd58a3d959 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:46.964598 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162aea3-5c00-4065-807c-57cd58a3d959 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:46.965127 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162aea3-5c00-4065-807c-57cd58a3d959 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:46.967886 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 485/953] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:09:46.974559 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087033e-89af-4f64-a55c-b824c647826e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:46.975500 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087033e-89af-4f64-a55c-b824c647826e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:46.976265 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087033e-89af-4f64-a55c-b824c647826e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:47.013105 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c087033e-89af-4f64-a55c-b824c647826e None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:47.016327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c087033e-89af-4f64-a55c-b824c647826e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:47.074657 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be29b7-f4bc-45ed-a6d3-065bf6de847f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:47.122377 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bdb5d9-9ca7-4cfb-b4ee-b575c221616b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:47.123133 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdb5d9-9ca7-4cfb-b4ee-b575c221616b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:47.123443 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdb5d9-9ca7-4cfb-b4ee-b575c221616b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:47.123850 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdb5d9-9ca7-4cfb-b4ee-b575c221616b None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:47.124619 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bdb5d9-9ca7-4cfb-b4ee-b575c221616b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:47.124936 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bdb5d9-9ca7-4cfb-b4ee-b575c221616b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:47.126362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bdb5d9-9ca7-4cfb-b4ee-b575c221616b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:47.139548 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 469/954] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Jul 25 19:09:46 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:09:47.144828 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bdb5d9-9ca7-4cfb-b4ee-b575c221616b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:47.162480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 486/955] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:09:47.191736 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087033e-89af-4f64-a55c-b824c647826e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:47.267685 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:47.329915 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bdb5d9-9ca7-4cfb-b4ee-b575c221616b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:47.364805 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:47.381536 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08760ba2-3b7d-4588-839a-74aa083bd1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:47.382689 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08760ba2-3b7d-4588-839a-74aa083bd1ed None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:47.383201 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08760ba2-3b7d-4588-839a-74aa083bd1ed None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:47.383751 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08760ba2-3b7d-4588-839a-74aa083bd1ed None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:47.384680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08760ba2-3b7d-4588-839a-74aa083bd1ed None admin] RBAC: Adding query filter params (type=compute) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:47.385792 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08760ba2-3b7d-4588-839a-74aa083bd1ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:47.387462 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08760ba2-3b7d-4588-839a-74aa083bd1ed None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:47.394393 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 470/956] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Jul 25 19:09:47 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:47.414444 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087033e-89af-4f64-a55c-b824c647826e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:47.473599 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61302d4d-40f9-4417-80b9-00239b0b5f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:47.474821 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61302d4d-40f9-4417-80b9-00239b0b5f87 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:47.475416 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61302d4d-40f9-4417-80b9-00239b0b5f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:47.475921 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61302d4d-40f9-4417-80b9-00239b0b5f87 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:47.476895 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61302d4d-40f9-4417-80b9-00239b0b5f87 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:47.479220 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61302d4d-40f9-4417-80b9-00239b0b5f87 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:47.499685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-61302d4d-40f9-4417-80b9-00239b0b5f87 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 19:09:47.499685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:47.499685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 19:09:47.499685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 19:09:47.499685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 19:09:47.499685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 19:09:47.499685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 19:09:47.499685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:47.499685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 19:09:47.499685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:47.499685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 19:09:47.499685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:47.499685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:47.499685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:47.499685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:47.499685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:47.499685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:47.501047 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:47.501047 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:47.501047 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:47.501047 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:47.501047 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:47.501047 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:47.501047 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 19:09:47.501047 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 19:09:47.501047 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 19:09:47.501047 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 19:09:47.501047 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:47.501047 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:47.501047 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:47.501047 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:47.501047 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:47.501047 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:47.502390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:47.502390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:47.502390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:47.502390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:47.502390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:47.502390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:47.502390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:47.502390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:47.502390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:47.502390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:47.502390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 19:09:47.502390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:47.502390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 19:09:47.502390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:47.505473 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 487/957] 10.222.0.18 () {56 vars in 1126 bytes} [Mon Jul 25 19:09:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:47.513993 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08760ba2-3b7d-4588-839a-74aa083bd1ed None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:47.663008 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08760ba2-3b7d-4588-839a-74aa083bd1ed None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:47.677214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:47.713383 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb3d71-6456-4f4f-82aa-e4034f4952b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:47.715055 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb3d71-6456-4f4f-82aa-e4034f4952b0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:47.715961 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb3d71-6456-4f4f-82aa-e4034f4952b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:47.716721 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb3d71-6456-4f4f-82aa-e4034f4952b0 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:47.717704 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb3d71-6456-4f4f-82aa-e4034f4952b0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:47.718256 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb3d71-6456-4f4f-82aa-e4034f4952b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:47.719999 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb3d71-6456-4f4f-82aa-e4034f4952b0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:47.728629 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 471/958] 10.222.0.18 () {56 vars in 1122 bytes} [Mon Jul 25 19:09:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:47.735557 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61302d4d-40f9-4417-80b9-00239b0b5f87 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:47.868682 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61302d4d-40f9-4417-80b9-00239b0b5f87 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:47.913459 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd18049-c702-4d61-a761-b4c487669e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:47.914550 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd18049-c702-4d61-a761-b4c487669e34 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:47.915097 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd18049-c702-4d61-a761-b4c487669e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:47.915627 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd18049-c702-4d61-a761-b4c487669e34 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:47.917196 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd18049-c702-4d61-a761-b4c487669e34 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:47.917796 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd18049-c702-4d61-a761-b4c487669e34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:47.919562 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd18049-c702-4d61-a761-b4c487669e34 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:47.926336 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 488/959] 10.222.0.18 () {56 vars in 1122 bytes} [Mon Jul 25 19:09:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:47.936691 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb3d71-6456-4f4f-82aa-e4034f4952b0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:48.005809 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b4150d-8c3e-432e-98c9-d8d98a82a420 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:48.006549 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b4150d-8c3e-432e-98c9-d8d98a82a420 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:48.007039 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b4150d-8c3e-432e-98c9-d8d98a82a420 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:48.009508 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 489/960] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:48.016671 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a46dc6-7328-4691-af5c-3e23e44cbb79 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:48.017326 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a46dc6-7328-4691-af5c-3e23e44cbb79 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:48.017823 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a46dc6-7328-4691-af5c-3e23e44cbb79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:48.042350 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-14a46dc6-7328-4691-af5c-3e23e44cbb79 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:48.044224 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-14a46dc6-7328-4691-af5c-3e23e44cbb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:48.049494 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:48.121936 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb3d71-6456-4f4f-82aa-e4034f4952b0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:48.171089 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 490/961] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 19:09:48.192704 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378e819-4992-4999-8e19-05853ba32e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:48.194327 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378e819-4992-4999-8e19-05853ba32e20 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:48.195092 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378e819-4992-4999-8e19-05853ba32e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:48.195896 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378e819-4992-4999-8e19-05853ba32e20 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:48.198200 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378e819-4992-4999-8e19-05853ba32e20 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:48.199443 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a46dc6-7328-4691-af5c-3e23e44cbb79 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:48.201381 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378e819-4992-4999-8e19-05853ba32e20 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:48.243089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 472/962] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:09:47 2022] POST /identity/v3/endpoints => generated 349 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:09:48.423047 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a46dc6-7328-4691-af5c-3e23e44cbb79 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:48.516013 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e1c6c4-b18c-4967-b554-a449f4ca9b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:48.517484 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1c6c4-b18c-4967-b554-a449f4ca9b1a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:48.517983 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1c6c4-b18c-4967-b554-a449f4ca9b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:48.518478 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1c6c4-b18c-4967-b554-a449f4ca9b1a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:48.524107 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:48.540546 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e1c6c4-b18c-4967-b554-a449f4ca9b1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:09:48.561740 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-33e1c6c4-b18c-4967-b554-a449f4ca9b1a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 19:09:48.561740 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:48.561740 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:48.561740 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:48.561740 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:48.561740 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:48.561740 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:48.561740 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:48.561740 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:48.561740 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:48.561740 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:48.561740 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:48.561740 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:09:48.561740 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:09:48.561740 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:09:48.561740 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:09:48.562831 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:09:48.562831 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:09:48.562831 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:09:48.562831 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:09:48.562831 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:48.562831 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:48.562831 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:48.562831 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:48.562831 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:48.562831 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:48.562831 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:48.562831 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:48.562831 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:48.562831 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:48.562831 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:48.562831 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:48.563867 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:48.563867 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:48.563867 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:48.563867 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:48.563867 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:09:48.563867 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:09:48.563867 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:09:48.563867 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:09:48.563867 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:09:48.563867 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:09:48.563867 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 19:09:48.563867 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:48.565085 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 491/963] 10.222.0.18 () {56 vars in 1120 bytes} [Mon Jul 25 19:09:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:09:48.576226 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378e819-4992-4999-8e19-05853ba32e20 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:48.643906 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:48.793030 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378e819-4992-4999-8e19-05853ba32e20 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:48.865762 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33413a8c-e4ba-4aa4-89ff-7992dc496a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:48.866470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33413a8c-e4ba-4aa4-89ff-7992dc496a08 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:48.866757 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33413a8c-e4ba-4aa4-89ff-7992dc496a08 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:48.867041 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33413a8c-e4ba-4aa4-89ff-7992dc496a08 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:48.867867 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33413a8c-e4ba-4aa4-89ff-7992dc496a08 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:48.868222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33413a8c-e4ba-4aa4-89ff-7992dc496a08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:48.869808 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33413a8c-e4ba-4aa4-89ff-7992dc496a08 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:48.880725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 473/964] 10.222.0.18 () {56 vars in 1115 bytes} [Mon Jul 25 19:09:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:48.890718 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e1c6c4-b18c-4967-b554-a449f4ca9b1a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:49.096571 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e1c6c4-b18c-4967-b554-a449f4ca9b1a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:49.138849 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ea802c-75a5-4767-ae24-b941ccef5081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:49.139719 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea802c-75a5-4767-ae24-b941ccef5081 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:49.140002 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea802c-75a5-4767-ae24-b941ccef5081 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:49.140291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea802c-75a5-4767-ae24-b941ccef5081 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:49.154266 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ea802c-75a5-4767-ae24-b941ccef5081 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:49.155935 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ea802c-75a5-4767-ae24-b941ccef5081 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:49.179393 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-69ea802c-75a5-4767-ae24-b941ccef5081 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:49.179393 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:49.179393 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:49.179393 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:49.179393 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:49.179393 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:49.179393 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:49.179393 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:49.179393 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:49.179393 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:49.179393 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:49.179393 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:49.179393 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:09:49.179393 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:09:49.179393 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:09:49.181171 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:09:49.181171 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:49.181171 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:49.181171 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:49.181171 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:49.181171 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:49.181171 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:49.181171 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:49.181171 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:49.181171 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:49.181171 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:49.181171 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:49.181171 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:49.181171 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:49.181171 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:49.181171 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:49.181171 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:49.182871 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:09:49.182871 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:09:49.182871 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:49.182871 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:49.184210 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 492/965] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:09:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:49.194272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33413a8c-e4ba-4aa4-89ff-7992dc496a08 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:49.386393 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:49.403972 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33413a8c-e4ba-4aa4-89ff-7992dc496a08 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:49.450152 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac35b7e-7ca1-4aea-a3b6-837db010ef6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:49.451251 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac35b7e-7ca1-4aea-a3b6-837db010ef6b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:49.451798 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac35b7e-7ca1-4aea-a3b6-837db010ef6b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:49.452382 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac35b7e-7ca1-4aea-a3b6-837db010ef6b None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:49.453357 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac35b7e-7ca1-4aea-a3b6-837db010ef6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:49.454127 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac35b7e-7ca1-4aea-a3b6-837db010ef6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:49.455900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac35b7e-7ca1-4aea-a3b6-837db010ef6b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:49.472148 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 474/966] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:09:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:49.481224 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69bc4b8-39c6-449c-9bca-e5a2a84dace9 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:49.481753 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69bc4b8-39c6-449c-9bca-e5a2a84dace9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:49.481831 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69bc4b8-39c6-449c-9bca-e5a2a84dace9 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:49.482134 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac35b7e-7ca1-4aea-a3b6-837db010ef6b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:49.483505 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 493/967] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:49.490808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8c5b1-c9ff-42b8-a8f7-2109a87624df None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:49.491174 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8c5b1-c9ff-42b8-a8f7-2109a87624df None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:49.491380 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8c5b1-c9ff-42b8-a8f7-2109a87624df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:49.515397 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d6e8c5b1-c9ff-42b8-a8f7-2109a87624df None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:49.516725 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d6e8c5b1-c9ff-42b8-a8f7-2109a87624df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:49.601100 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:49.629670 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac35b7e-7ca1-4aea-a3b6-837db010ef6b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:49.666591 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 494/968] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 19:09:49.693641 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8c5b1-c9ff-42b8-a8f7-2109a87624df None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:49.696785 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f74321-923d-477a-988d-3dbdff86cd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:49.699302 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f74321-923d-477a-988d-3dbdff86cd46 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:49.700021 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f74321-923d-477a-988d-3dbdff86cd46 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:49.700568 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f74321-923d-477a-988d-3dbdff86cd46 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:49.710993 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f74321-923d-477a-988d-3dbdff86cd46 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:49.713479 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f74321-923d-477a-988d-3dbdff86cd46 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:49.733895 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-31f74321-923d-477a-988d-3dbdff86cd46 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 19:09:49.733895 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:49.733895 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:49.733895 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:49.733895 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:49.733895 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:49.733895 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:49.733895 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:49.733895 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:49.733895 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:49.733895 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:49.733895 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:49.733895 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:09:49.733895 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:09:49.733895 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:09:49.733895 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:09:49.735085 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:49.735085 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:49.735085 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:09:49.735085 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:09:49.735085 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:09:49.735085 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:09:49.735085 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:49.735085 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:49.735085 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:49.735085 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:49.735085 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:49.735085 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:49.735085 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:49.735085 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:49.735085 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:49.735085 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:49.735085 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:49.735085 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:49.736139 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:49.736139 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:49.736139 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:09:49.736139 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:09:49.736139 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:09:49.736139 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:09:49.736139 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:09:49.736139 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:09:49.736139 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 19:09:49.736139 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:49.739059 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 475/969] 10.222.0.18 () {56 vars in 1117 bytes} [Mon Jul 25 19:09:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:49.751106 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f74321-923d-477a-988d-3dbdff86cd46 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:49.774566 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:49.884831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f74321-923d-477a-988d-3dbdff86cd46 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:49.905346 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8c5b1-c9ff-42b8-a8f7-2109a87624df None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:49.925671 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f5879-4388-4062-a839-8d8817cdd5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:49.926288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f5879-4388-4062-a839-8d8817cdd5b8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:49.926575 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f5879-4388-4062-a839-8d8817cdd5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:49.926847 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f5879-4388-4062-a839-8d8817cdd5b8 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:49.927747 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f5879-4388-4062-a839-8d8817cdd5b8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:49.928061 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f5879-4388-4062-a839-8d8817cdd5b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:49.929572 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f5879-4388-4062-a839-8d8817cdd5b8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:49.952871 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 476/970] 10.222.0.18 () {56 vars in 1149 bytes} [Mon Jul 25 19:09:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:09:49.964138 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f5879-4388-4062-a839-8d8817cdd5b8 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:49.980701 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8772bd-6de8-49e7-b9a4-0d9f422fb5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:49.981396 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8772bd-6de8-49e7-b9a4-0d9f422fb5ce None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:49.981678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8772bd-6de8-49e7-b9a4-0d9f422fb5ce None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:49.981955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8772bd-6de8-49e7-b9a4-0d9f422fb5ce None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:49.982686 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8772bd-6de8-49e7-b9a4-0d9f422fb5ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:49.984270 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8772bd-6de8-49e7-b9a4-0d9f422fb5ce None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:50.005593 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.sql.core [None req-be8772bd-6de8-49e7-b9a4-0d9f422fb5ce None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 25 19:09:50.005593 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 25 19:09:50.005593 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [parameters: {'user_id': 'eeb478952adf419aaf6c0f3a4c69e193', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 25 19:09:50.005593 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80237) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 25 19:09:50.015273 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-be8772bd-6de8-49e7-b9a4-0d9f422fb5ce None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 25 19:09:50.015273 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:50.015273 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 19:09:50.015273 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 19:09:50.015273 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 19:09:50.015273 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 19:09:50.015273 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 19:09:50.015273 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 19:09:50.015273 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 19:09:50.015273 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application conn.query(q) Jul 25 19:09:50.015273 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 19:09:50.015273 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 19:09:50.015273 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 19:09:50.015273 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result.read() Jul 25 19:09:50.015273 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 19:09:50.016267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 19:09:50.016267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 19:09:50.016267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 19:09:50.016267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 19:09:50.016267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 19:09:50.016267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 19:09:50.016267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 19:09:50.016267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 25 19:09:50.016267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:50.016267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 25 19:09:50.016267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:50.016267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:50.016267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 25 19:09:50.016267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 25 19:09:50.016267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 25 19:09:50.016267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 25 19:09:50.016267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 19:09:50.016267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application next(self.gen) Jul 25 19:09:50.017247 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 25 19:09:50.017247 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application yield resource Jul 25 19:09:50.017247 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 19:09:50.017247 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application next(self.gen) Jul 25 19:09:50.017247 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 25 19:09:50.017247 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.session.rollback() Jul 25 19:09:50.017247 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 25 19:09:50.017247 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.force_reraise() Jul 25 19:09:50.017247 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 25 19:09:50.017247 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise self.value Jul 25 19:09:50.017247 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 25 19:09:50.017247 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 25 19:09:50.017247 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 25 19:09:50.017247 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application session.commit() Jul 25 19:09:50.017247 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 25 19:09:50.017247 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 25 19:09:50.017247 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 25 19:09:50.018188 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._prepare_impl() Jul 25 19:09:50.018188 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 25 19:09:50.018188 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.session.flush() Jul 25 19:09:50.018188 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 25 19:09:50.018188 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._flush(objects) Jul 25 19:09:50.018188 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 25 19:09:50.018188 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 25 19:09:50.018188 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 25 19:09:50.018188 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application compat.raise_( Jul 25 19:09:50.018188 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 19:09:50.018188 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception Jul 25 19:09:50.018188 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 25 19:09:50.018188 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application flush_context.execute() Jul 25 19:09:50.018188 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 25 19:09:50.018188 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rec.execute(self) Jul 25 19:09:50.018188 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 25 19:09:50.018188 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 25 19:09:50.019227 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 25 19:09:50.019227 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 25 19:09:50.019227 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 25 19:09:50.019227 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 25 19:09:50.019227 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 25 19:09:50.019227 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 25 19:09:50.019227 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 25 19:09:50.019227 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 25 19:09:50.019227 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 25 19:09:50.019227 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 25 19:09:50.019227 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 25 19:09:50.019227 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 25 19:09:50.019227 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 25 19:09:50.019227 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 25 19:09:50.019227 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 19:09:50.019227 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception Jul 25 19:09:50.021879 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 19:09:50.021879 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 19:09:50.021879 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 19:09:50.021879 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 19:09:50.021879 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 19:09:50.021879 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 19:09:50.021879 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 19:09:50.021879 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application conn.query(q) Jul 25 19:09:50.021879 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 19:09:50.021879 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 19:09:50.021879 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 19:09:50.021879 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result.read() Jul 25 19:09:50.021879 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 19:09:50.021879 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 19:09:50.021879 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 19:09:50.021879 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 19:09:50.023291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 19:09:50.023291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 19:09:50.023291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 19:09:50.023291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 19:09:50.023291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 25 19:09:50.023291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 25 19:09:50.023291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [parameters: {'user_id': 'eeb478952adf419aaf6c0f3a4c69e193', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 25 19:09:50.023291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 25 19:09:50.023291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:50.023291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 19:09:50.023291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:50.023291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:50.023291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:50.023291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:50.023291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:50.028659 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:50.028659 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:50.028659 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:50.028659 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:50.028659 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:50.028659 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:50.028659 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:50.028659 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 25 19:09:50.028659 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 25 19:09:50.028659 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:50.028659 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:50.028659 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:09:50.028659 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:09:50.028659 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:09:50.028659 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:09:50.028659 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 25 19:09:50.028659 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 25 19:09:50.029721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 25 19:09:50.029721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 25 19:09:50.029721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 25 19:09:50.029721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 25 19:09:50.029721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 25 19:09:50.029721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:50.029721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 495/971] 10.222.0.18 () {60 vars in 1131 bytes} [Mon Jul 25 19:09:49 2022] POST /identity/v3/users => generated 160 bytes in 325 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:50.029721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8772bd-6de8-49e7-b9a4-0d9f422fb5ce None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:50.112669 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:50.115346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:50.164386 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f5879-4388-4062-a839-8d8817cdd5b8 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:50.236074 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d7ddf-13d7-440c-99f1-b4ba3e2a6f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:50.237062 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d7ddf-13d7-440c-99f1-b4ba3e2a6f42 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:50.237522 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d7ddf-13d7-440c-99f1-b4ba3e2a6f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:50.237988 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d7ddf-13d7-440c-99f1-b4ba3e2a6f42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:50.240953 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8772bd-6de8-49e7-b9a4-0d9f422fb5ce None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:50.310113 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13036d58-4933-4bfd-b5fa-b944be42c58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:50.311498 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13036d58-4933-4bfd-b5fa-b944be42c58c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:50.312182 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13036d58-4933-4bfd-b5fa-b944be42c58c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:50.312543 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13036d58-4933-4bfd-b5fa-b944be42c58c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:50.328268 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13036d58-4933-4bfd-b5fa-b944be42c58c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:50.331341 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13036d58-4933-4bfd-b5fa-b944be42c58c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:50.353189 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-13036d58-4933-4bfd-b5fa-b944be42c58c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 19:09:50.353189 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:50.353189 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:50.353189 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:50.353189 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:50.353189 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:50.353189 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:50.353189 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:50.353189 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:50.353189 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:50.353189 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:50.353189 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:50.353189 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:09:50.353189 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:09:50.353189 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:09:50.353189 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:09:50.354849 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:50.354849 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:50.354849 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:09:50.354849 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:09:50.354849 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:09:50.354849 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:09:50.354849 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:50.354849 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:50.354849 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:50.354849 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:50.354849 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:50.354849 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:50.354849 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:50.354849 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:50.354849 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:50.354849 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:50.354849 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:50.354849 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:50.356657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:50.356657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:50.356657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:09:50.356657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:09:50.356657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:09:50.356657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:09:50.356657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:09:50.356657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:09:50.356657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 19:09:50.356657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:50.356657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 496/972] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:09:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:50.357315 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cab4477-774f-49c6-93e1-db2a1d57bdd4 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:50.357315 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cab4477-774f-49c6-93e1-db2a1d57bdd4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:50.357767 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cab4477-774f-49c6-93e1-db2a1d57bdd4 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:50.359203 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d7ddf-13d7-440c-99f1-b4ba3e2a6f42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:50.361102 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d7ddf-13d7-440c-99f1-b4ba3e2a6f42 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:50.364815 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 497/973] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:50.364815 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cab4477-774f-49c6-93e1-db2a1d57bdd4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:50.455183 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:50.512896 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 477/974] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:09:49 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 19:09:50.512896 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a391cf-333a-42fb-9c26-e0ce87ca7a02 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:50.512896 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a391cf-333a-42fb-9c26-e0ce87ca7a02 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:50.512896 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a391cf-333a-42fb-9c26-e0ce87ca7a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:50.518426 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:50.527656 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cab4477-774f-49c6-93e1-db2a1d57bdd4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:50.529339 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f8a391cf-333a-42fb-9c26-e0ce87ca7a02 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:50.530619 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f8a391cf-333a-42fb-9c26-e0ce87ca7a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:50.579620 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28653128-d81c-45f2-96d3-af530c1f77fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:50.580323 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28653128-d81c-45f2-96d3-af530c1f77fe None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:50.580684 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28653128-d81c-45f2-96d3-af530c1f77fe None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:50.580966 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28653128-d81c-45f2-96d3-af530c1f77fe None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:50.581804 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28653128-d81c-45f2-96d3-af530c1f77fe None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:50.582224 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28653128-d81c-45f2-96d3-af530c1f77fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:50.583867 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28653128-d81c-45f2-96d3-af530c1f77fe None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:50.628070 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 498/975] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Jul 25 19:09:50 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:50.629008 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28653128-d81c-45f2-96d3-af530c1f77fe None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:50.685861 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 478/976] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:09:50.731811 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a391cf-333a-42fb-9c26-e0ce87ca7a02 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:50.825673 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28653128-d81c-45f2-96d3-af530c1f77fe None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:50.875335 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:50.881201 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a391cf-333a-42fb-9c26-e0ce87ca7a02 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:50.886189 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddcc136-4858-4764-90e5-a8d8c2189e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:50.887055 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddcc136-4858-4764-90e5-a8d8c2189e90 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:50.887363 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddcc136-4858-4764-90e5-a8d8c2189e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:50.889125 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddcc136-4858-4764-90e5-a8d8c2189e90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:50.908809 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddcc136-4858-4764-90e5-a8d8c2189e90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:50.911614 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddcc136-4858-4764-90e5-a8d8c2189e90 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:50.941289 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-0ddcc136-4858-4764-90e5-a8d8c2189e90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:50.941289 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:50.941289 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:50.941289 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:50.941289 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:50.941289 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:50.941289 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:50.941289 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:50.941289 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:50.941289 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:50.941289 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:50.941289 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:50.941289 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:09:50.941289 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:09:50.941289 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:09:50.943101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:09:50.943101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:50.943101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:50.943101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:50.943101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:50.943101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:50.943101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:50.943101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:50.943101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:50.943101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:50.943101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:50.943101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:50.943101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:50.943101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:50.943101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:50.943101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:50.943101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:50.948533 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:09:50.948533 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:09:50.948533 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:09:50.948533 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:50.948533 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 499/977] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:09:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:50.949909 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123f3cc-56f2-4dfc-8821-7e516fb8a5ae None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:50.950274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123f3cc-56f2-4dfc-8821-7e516fb8a5ae None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:50.950598 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123f3cc-56f2-4dfc-8821-7e516fb8a5ae None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:50.952310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 500/978] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:50.953042 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0123f3cc-56f2-4dfc-8821-7e516fb8a5ae None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:50.953226 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6bc523-92b2-420b-beda-2103003755be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:50.954308 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6bc523-92b2-420b-beda-2103003755be None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:50.954788 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6bc523-92b2-420b-beda-2103003755be None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:50.955242 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6bc523-92b2-420b-beda-2103003755be None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:50.957914 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6bc523-92b2-420b-beda-2103003755be None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:50.958967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6bc523-92b2-420b-beda-2103003755be None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:50.972866 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:50.982640 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 479/979] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Jul 25 19:09:50 2022] POST /identity/v3/services => generated 242 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:09:50.984027 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0439b7e-1cb2-46e3-bf5e-6734bdffd3fd None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:50.984341 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0439b7e-1cb2-46e3-bf5e-6734bdffd3fd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:50.984613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0439b7e-1cb2-46e3-bf5e-6734bdffd3fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:51.012613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b0439b7e-1cb2-46e3-bf5e-6734bdffd3fd None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:51.014469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b0439b7e-1cb2-46e3-bf5e-6734bdffd3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:51.116983 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 480/980] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:50 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:09:51.129222 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0123f3cc-56f2-4dfc-8821-7e516fb8a5ae None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:51.145408 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0439b7e-1cb2-46e3-bf5e-6734bdffd3fd None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:51.188300 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77788e01-bb6e-4b14-b6f2-fe06b163587c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:51.189006 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77788e01-bb6e-4b14-b6f2-fe06b163587c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:51.189296 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77788e01-bb6e-4b14-b6f2-fe06b163587c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:51.189576 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77788e01-bb6e-4b14-b6f2-fe06b163587c None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:51.190863 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77788e01-bb6e-4b14-b6f2-fe06b163587c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:51.191141 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77788e01-bb6e-4b14-b6f2-fe06b163587c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:51.192794 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77788e01-bb6e-4b14-b6f2-fe06b163587c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:51.209054 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 501/981] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:51.215867 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77788e01-bb6e-4b14-b6f2-fe06b163587c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:51.280795 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:51.301901 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0439b7e-1cb2-46e3-bf5e-6734bdffd3fd None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:51.356606 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7320c862-8249-4f0b-af38-b2c4fd2281d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:51.357405 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:51.357593 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320c862-8249-4f0b-af38-b2c4fd2281d6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:51.357752 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320c862-8249-4f0b-af38-b2c4fd2281d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:51.358003 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320c862-8249-4f0b-af38-b2c4fd2281d6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:51.358692 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7320c862-8249-4f0b-af38-b2c4fd2281d6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:51.360846 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7320c862-8249-4f0b-af38-b2c4fd2281d6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:51.376485 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-7320c862-8249-4f0b-af38-b2c4fd2281d6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 19:09:51.376485 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:51.376485 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 19:09:51.376485 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 19:09:51.376485 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 19:09:51.376485 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 19:09:51.376485 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 19:09:51.376485 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:51.376485 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 19:09:51.376485 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:51.376485 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 19:09:51.376485 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:51.376485 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:51.376485 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:51.376485 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:51.376485 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:51.376485 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:51.379314 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:51.379314 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:51.379314 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:51.379314 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:51.379314 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:51.379314 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:51.379314 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 19:09:51.379314 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 19:09:51.379314 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 19:09:51.379314 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 19:09:51.379314 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:51.379314 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:51.379314 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:51.379314 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:51.379314 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:51.379314 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:51.381841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:51.381841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:51.381841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:51.381841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:51.381841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:51.381841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:51.381841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:51.381841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:51.381841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:51.381841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:51.381841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 19:09:51.381841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:51.381841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 19:09:51.381841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:51.387329 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 481/982] 10.222.0.18 () {56 vars in 1126 bytes} [Mon Jul 25 19:09:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:09:51.387329 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7320c862-8249-4f0b-af38-b2c4fd2281d6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:51.415873 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77788e01-bb6e-4b14-b6f2-fe06b163587c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:51.463638 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095addc-ecd5-4a45-9b43-21943ce645f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:51.464506 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095addc-ecd5-4a45-9b43-21943ce645f0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:51.464803 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095addc-ecd5-4a45-9b43-21943ce645f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:51.465073 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095addc-ecd5-4a45-9b43-21943ce645f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:51.474152 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d095addc-ecd5-4a45-9b43-21943ce645f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:51.476022 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d095addc-ecd5-4a45-9b43-21943ce645f0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:51.495488 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d095addc-ecd5-4a45-9b43-21943ce645f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:09:51.495488 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:51.495488 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:51.495488 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:51.495488 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:51.495488 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:51.495488 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:51.495488 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:51.495488 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:51.495488 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:51.495488 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:51.495488 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:51.495488 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:09:51.495488 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:09:51.495488 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:09:51.497720 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:09:51.497720 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:51.497720 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:51.497720 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:51.497720 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:51.497720 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:51.497720 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:51.497720 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:51.497720 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:51.497720 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:51.497720 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:51.497720 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:51.497720 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:51.497720 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:51.497720 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:51.497720 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:51.497720 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:51.499943 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:09:51.499943 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:09:51.499943 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:09:51.499943 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:09:51.499943 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:09:51.499943 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:09:51.499943 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:09:51.499943 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:51.499943 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 502/983] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:09:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:51.503114 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095addc-ecd5-4a45-9b43-21943ce645f0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:51.566159 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7320c862-8249-4f0b-af38-b2c4fd2281d6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:51.613900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823aec1d-b175-4b55-9f6d-bd724f9e1d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:51.614598 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823aec1d-b175-4b55-9f6d-bd724f9e1d15 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:51.614882 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823aec1d-b175-4b55-9f6d-bd724f9e1d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:51.615156 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823aec1d-b175-4b55-9f6d-bd724f9e1d15 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:51.615955 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823aec1d-b175-4b55-9f6d-bd724f9e1d15 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:51.616261 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823aec1d-b175-4b55-9f6d-bd724f9e1d15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:51.617774 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823aec1d-b175-4b55-9f6d-bd724f9e1d15 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:51.629183 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 482/984] 10.222.0.18 () {56 vars in 1122 bytes} [Mon Jul 25 19:09:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:51.631698 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823aec1d-b175-4b55-9f6d-bd724f9e1d15 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:51.652186 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:51.673915 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095addc-ecd5-4a45-9b43-21943ce645f0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:51.731167 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:51.741706 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bafc83c-763d-4ea7-a98d-46d89a6965a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:51.742500 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafc83c-763d-4ea7-a98d-46d89a6965a6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:51.742792 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafc83c-763d-4ea7-a98d-46d89a6965a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:51.743077 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafc83c-763d-4ea7-a98d-46d89a6965a6 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:51.743922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bafc83c-763d-4ea7-a98d-46d89a6965a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:51.744873 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bafc83c-763d-4ea7-a98d-46d89a6965a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:51.746475 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bafc83c-763d-4ea7-a98d-46d89a6965a6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:51.761316 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 503/985] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Jul 25 19:09:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:09:51.767323 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bafc83c-763d-4ea7-a98d-46d89a6965a6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:51.788561 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823aec1d-b175-4b55-9f6d-bd724f9e1d15 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:51.840955 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a4a7c-84d8-48d8-acaa-67a295837130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:51.841915 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a4a7c-84d8-48d8-acaa-67a295837130 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:51.842210 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a4a7c-84d8-48d8-acaa-67a295837130 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:51.842464 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a4a7c-84d8-48d8-acaa-67a295837130 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:51.843191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a4a7c-84d8-48d8-acaa-67a295837130 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:51.843499 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a4a7c-84d8-48d8-acaa-67a295837130 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:51.845340 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a4a7c-84d8-48d8-acaa-67a295837130 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:51.851732 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 483/986] 10.222.0.18 () {56 vars in 1122 bytes} [Mon Jul 25 19:09:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:09:51.859157 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a4a7c-84d8-48d8-acaa-67a295837130 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:51.914051 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bafc83c-763d-4ea7-a98d-46d89a6965a6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:51.962435 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c6e46-209d-4fe7-835b-436d6c673d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:51.963988 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c6e46-209d-4fe7-835b-436d6c673d98 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:51.964225 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c6e46-209d-4fe7-835b-436d6c673d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:51.964469 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c6e46-209d-4fe7-835b-436d6c673d98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:51.965412 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40c6e46-209d-4fe7-835b-436d6c673d98 None admin] RBAC: Adding query filter params (role.id=e1dc00443f96426e9edf6ea6ca8ae542, scope.project.id=191069caccd04233b012585e29cfa18b, user.id=1c0772205ae24713b8901666a87018b2) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:51.965688 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40c6e46-209d-4fe7-835b-436d6c673d98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:51.967483 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40c6e46-209d-4fe7-835b-436d6c673d98 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:51.972177 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:51.976486 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 504/987] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:09:51 2022] GET /identity/v3/role_assignments?user.id=1c0772205ae24713b8901666a87018b2&scope.project.id=191069caccd04233b012585e29cfa18b&role.id=e1dc00443f96426e9edf6ea6ca8ae542 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:09:52.013844 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a4a7c-84d8-48d8-acaa-67a295837130 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:52.068403 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fb7ef8-77e7-453f-b823-303bf1c4e70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:52.069444 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb7ef8-77e7-453f-b823-303bf1c4e70b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:52.069951 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb7ef8-77e7-453f-b823-303bf1c4e70b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:52.070419 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb7ef8-77e7-453f-b823-303bf1c4e70b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:52.071463 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fb7ef8-77e7-453f-b823-303bf1c4e70b None admin] RBAC: Adding query filter params (interface=admin, service_id=ebda878503be465e98903768e3143079, region_id=RegionOne) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:52.071978 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fb7ef8-77e7-453f-b823-303bf1c4e70b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:52.074801 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fb7ef8-77e7-453f-b823-303bf1c4e70b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:52.086226 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 484/988] 10.222.0.18 () {56 vars in 1249 bytes} [Mon Jul 25 19:09:51 2022] GET /identity/v3/endpoints?service_id=ebda878503be465e98903768e3143079&interface=admin®ion_id=RegionOne => generated 193 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:09:52.087620 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c6e46-209d-4fe7-835b-436d6c673d98 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:52.155712 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:52.234355 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c6e46-209d-4fe7-835b-436d6c673d98 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:52.289050 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eae58cb-85f6-43e8-a389-71b2c94f9cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:52.289821 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eae58cb-85f6-43e8-a389-71b2c94f9cb8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:52.290111 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eae58cb-85f6-43e8-a389-71b2c94f9cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:52.290396 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eae58cb-85f6-43e8-a389-71b2c94f9cb8 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:52.291848 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eae58cb-85f6-43e8-a389-71b2c94f9cb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:52.293238 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eae58cb-85f6-43e8-a389-71b2c94f9cb8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:52.301475 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 505/989] 10.222.0.18 () {56 vars in 1087 bytes} [Mon Jul 25 19:09:52 2022] GET /identity/v3/services => generated 1727 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:09:54.204883 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d8817-c358-40f6-942e-70f663cf7fe4 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:54.205826 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d8817-c358-40f6-942e-70f663cf7fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:54.206096 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d8817-c358-40f6-942e-70f663cf7fe4 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:54.208590 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 485/990] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:54.218360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff19127-883e-49d4-b7d5-1419f75ca589 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:54.219138 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff19127-883e-49d4-b7d5-1419f75ca589 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:54.219446 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff19127-883e-49d4-b7d5-1419f75ca589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:54.252267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-aff19127-883e-49d4-b7d5-1419f75ca589 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:54.255170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-aff19127-883e-49d4-b7d5-1419f75ca589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:54.398109 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 506/991] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:54 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:09:54.439022 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d8817-c358-40f6-942e-70f663cf7fe4 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:54.485644 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a3400-e09e-4221-8da4-eb7983d209d0 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:54.485952 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a3400-e09e-4221-8da4-eb7983d209d0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:54.486280 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a3400-e09e-4221-8da4-eb7983d209d0 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:54.491849 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 507/992] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:09:54.496455 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16170f52-9740-4961-92a0-37fef57d8bf8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:54.496865 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16170f52-9740-4961-92a0-37fef57d8bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:54.497488 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16170f52-9740-4961-92a0-37fef57d8bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:54.520690 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-16170f52-9740-4961-92a0-37fef57d8bf8 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:54.522087 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-16170f52-9740-4961-92a0-37fef57d8bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:54.593536 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:54.626646 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d8817-c358-40f6-942e-70f663cf7fe4 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:54.642340 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 508/993] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:54 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 19:09:54.644563 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d279695-8805-48f5-8261-1453077578d4 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:54.645064 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d279695-8805-48f5-8261-1453077578d4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:54.645569 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d279695-8805-48f5-8261-1453077578d4 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:54.648324 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 509/994] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:54.657960 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f449a-b5a0-4a5d-bd38-f301b4bb14ca None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:54.658479 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f449a-b5a0-4a5d-bd38-f301b4bb14ca None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:54.658972 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f449a-b5a0-4a5d-bd38-f301b4bb14ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:54.677714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3186f8-874f-42da-9b2f-14b37988d1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:54.678538 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3186f8-874f-42da-9b2f-14b37988d1eb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:54.678830 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3186f8-874f-42da-9b2f-14b37988d1eb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:54.679109 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3186f8-874f-42da-9b2f-14b37988d1eb None admin] PATH_INFO: `/v3/services/compute` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:54.679937 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3186f8-874f-42da-9b2f-14b37988d1eb None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:54.682210 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3186f8-874f-42da-9b2f-14b37988d1eb None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:54.691940 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e63f449a-b5a0-4a5d-bd38-f301b4bb14ca None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:54.693866 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e63f449a-b5a0-4a5d-bd38-f301b4bb14ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:54.696275 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-ab3186f8-874f-42da-9b2f-14b37988d1eb None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 19:09:54.696275 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:54.696275 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 19:09:54.696275 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 19:09:54.696275 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 19:09:54.696275 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 19:09:54.696275 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 19:09:54.696275 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:54.696275 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 19:09:54.696275 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:54.696275 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 19:09:54.696275 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:54.696275 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:54.696275 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:54.696275 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:54.696275 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:54.696275 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:54.699486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:54.699486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:54.699486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:54.699486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:54.699486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:54.699486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:54.699486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 19:09:54.699486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 19:09:54.699486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 19:09:54.699486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 19:09:54.699486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:54.699486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:54.699486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:54.699486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:54.699486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:54.699486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:54.701507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:54.701507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:54.701507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:54.701507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:54.701507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:54.701507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:54.701507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:54.701507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:54.701507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:54.701507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:54.701507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 19:09:54.701507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:54.701507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 19:09:54.701507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:54.701507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 486/995] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:09:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:54.703305 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3186f8-874f-42da-9b2f-14b37988d1eb None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:54.716434 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:54.801668 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 510/996] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:54 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 19:09:54.802896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f449a-b5a0-4a5d-bd38-f301b4bb14ca None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:54.878818 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3186f8-874f-42da-9b2f-14b37988d1eb None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:54.940529 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687237fd-74c1-43af-af08-dab5bd5d2ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:54.941781 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687237fd-74c1-43af-af08-dab5bd5d2ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:54.942309 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687237fd-74c1-43af-af08-dab5bd5d2ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:54.942693 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687237fd-74c1-43af-af08-dab5bd5d2ca8 None admin] PATH_INFO: `/v3/services/image` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:54.944035 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687237fd-74c1-43af-af08-dab5bd5d2ca8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:54.946283 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687237fd-74c1-43af-af08-dab5bd5d2ca8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:54.960856 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-687237fd-74c1-43af-af08-dab5bd5d2ca8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 19:09:54.960856 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:54.960856 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 19:09:54.960856 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 19:09:54.960856 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 19:09:54.960856 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 19:09:54.960856 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 19:09:54.960856 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:54.960856 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 19:09:54.960856 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:54.960856 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 19:09:54.960856 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:54.960856 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:54.960856 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:54.960856 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:54.960856 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:54.960856 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:54.964162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:54.964162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:54.964162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:54.964162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:54.964162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:54.964162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:54.964162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 19:09:54.964162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 19:09:54.964162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 19:09:54.964162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 19:09:54.964162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:54.964162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:54.964162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:54.964162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:54.964162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:54.964162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:54.965862 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:54.965862 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:54.965862 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:54.965862 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:54.965862 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:54.965862 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:54.965862 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:54.965862 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:54.965862 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:54.965862 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:54.965862 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 19:09:54.965862 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:54.965862 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 19:09:54.965862 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:54.965862 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 487/997] 10.222.0.18 () {56 vars in 1105 bytes} [Mon Jul 25 19:09:54 2022] GET /identity/v3/services/image => generated 86 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:54.967362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687237fd-74c1-43af-af08-dab5bd5d2ca8 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:54.972864 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f449a-b5a0-4a5d-bd38-f301b4bb14ca None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:54.982796 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:55.021976 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0af3c7-663b-48c7-9307-e5e22721ec97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:55.022752 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0af3c7-663b-48c7-9307-e5e22721ec97 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:55.023134 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0af3c7-663b-48c7-9307-e5e22721ec97 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:55.023471 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0af3c7-663b-48c7-9307-e5e22721ec97 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:55.024399 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0af3c7-663b-48c7-9307-e5e22721ec97 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:55.024775 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0af3c7-663b-48c7-9307-e5e22721ec97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:55.027359 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0af3c7-663b-48c7-9307-e5e22721ec97 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:55.033467 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 511/998] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Jul 25 19:09:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:55.034522 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0af3c7-663b-48c7-9307-e5e22721ec97 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:55.081249 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:55.132056 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687237fd-74c1-43af-af08-dab5bd5d2ca8 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:55.184230 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3054473-11f0-4a0b-be11-4eb5dc168e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:55.185060 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3054473-11f0-4a0b-be11-4eb5dc168e3f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:55.185344 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3054473-11f0-4a0b-be11-4eb5dc168e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:55.185622 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3054473-11f0-4a0b-be11-4eb5dc168e3f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:55.185787 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0af3c7-663b-48c7-9307-e5e22721ec97 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:55.186285 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3054473-11f0-4a0b-be11-4eb5dc168e3f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:55.187911 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3054473-11f0-4a0b-be11-4eb5dc168e3f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:55.210398 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-c3054473-11f0-4a0b-be11-4eb5dc168e3f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 19:09:55.210398 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:55.210398 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 19:09:55.210398 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 19:09:55.210398 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 19:09:55.210398 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 19:09:55.210398 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 19:09:55.210398 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:55.210398 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 19:09:55.210398 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:55.210398 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 19:09:55.210398 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:55.210398 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:55.210398 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:55.210398 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:55.210398 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:55.210398 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:55.212157 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:55.212157 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:55.212157 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:55.212157 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:55.212157 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:55.212157 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:55.212157 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 19:09:55.212157 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 19:09:55.212157 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 19:09:55.212157 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 19:09:55.212157 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:55.212157 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:55.212157 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:55.212157 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:55.212157 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:55.212157 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:55.214288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:55.214288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:55.214288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:55.214288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:55.214288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:55.214288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:55.214288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:55.214288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:55.214288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:55.214288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:55.214288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 19:09:55.214288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:55.214288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 19:09:55.214288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:09:55.214288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 488/999] 10.222.0.18 () {56 vars in 1126 bytes} [Mon Jul 25 19:09:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:55.215278 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3054473-11f0-4a0b-be11-4eb5dc168e3f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:55.241731 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9355ed1-c3fb-4e7c-836a-b91930e20efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:55.242880 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9355ed1-c3fb-4e7c-836a-b91930e20efc None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:55.243400 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9355ed1-c3fb-4e7c-836a-b91930e20efc None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:55.243939 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9355ed1-c3fb-4e7c-836a-b91930e20efc None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:55.245285 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9355ed1-c3fb-4e7c-836a-b91930e20efc None admin] RBAC: Adding query filter params (name=image) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:55.245818 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9355ed1-c3fb-4e7c-836a-b91930e20efc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:55.248539 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9355ed1-c3fb-4e7c-836a-b91930e20efc None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:55.257938 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 512/1000] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:09:55 2022] GET /identity/v3/services?name=image => generated 122 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:55.259615 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9355ed1-c3fb-4e7c-836a-b91930e20efc None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:55.310500 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:55.382257 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3054473-11f0-4a0b-be11-4eb5dc168e3f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:55.423235 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9355ed1-c3fb-4e7c-836a-b91930e20efc None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:55.438921 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801e581f-795e-4bb0-b887-7c2f8f254626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:55.440123 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e581f-795e-4bb0-b887-7c2f8f254626 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:55.440638 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e581f-795e-4bb0-b887-7c2f8f254626 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:55.441110 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e581f-795e-4bb0-b887-7c2f8f254626 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:55.442485 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801e581f-795e-4bb0-b887-7c2f8f254626 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:55.443052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801e581f-795e-4bb0-b887-7c2f8f254626 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:55.445755 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801e581f-795e-4bb0-b887-7c2f8f254626 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:55.448800 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:55.463294 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 489/1001] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Jul 25 19:09:55 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:55.463294 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801e581f-795e-4bb0-b887-7c2f8f254626 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:55.475598 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d082c596-e8e0-4df9-82a1-5295b50b74db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:55.476303 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082c596-e8e0-4df9-82a1-5295b50b74db None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:55.476591 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082c596-e8e0-4df9-82a1-5295b50b74db None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:55.476875 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082c596-e8e0-4df9-82a1-5295b50b74db None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:55.477605 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d082c596-e8e0-4df9-82a1-5295b50b74db None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:55.477913 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d082c596-e8e0-4df9-82a1-5295b50b74db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:55.479899 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d082c596-e8e0-4df9-82a1-5295b50b74db None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:55.486239 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 513/1002] 10.222.0.18 () {56 vars in 1122 bytes} [Mon Jul 25 19:09:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:55.487319 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d082c596-e8e0-4df9-82a1-5295b50b74db None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:55.589661 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801e581f-795e-4bb0-b887-7c2f8f254626 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:55.616514 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:55.621695 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d082c596-e8e0-4df9-82a1-5295b50b74db None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:55.632582 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df81c22f-4fd6-4ccb-9a5b-db7a5174e8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:55.633272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df81c22f-4fd6-4ccb-9a5b-db7a5174e8ef None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:55.633972 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df81c22f-4fd6-4ccb-9a5b-db7a5174e8ef None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:55.633972 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df81c22f-4fd6-4ccb-9a5b-db7a5174e8ef None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:55.635263 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df81c22f-4fd6-4ccb-9a5b-db7a5174e8ef None admin] RBAC: Adding query filter params (type=image) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:55.635648 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df81c22f-4fd6-4ccb-9a5b-db7a5174e8ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:55.637419 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df81c22f-4fd6-4ccb-9a5b-db7a5174e8ef None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:55.642892 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 490/1003] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:09:55 2022] GET /identity/v3/services?type=image => generated 122 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:55.643874 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df81c22f-4fd6-4ccb-9a5b-db7a5174e8ef None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:55.668770 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e30f1-c28d-4587-bfa7-40645b8a446a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:55.669467 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e30f1-c28d-4587-bfa7-40645b8a446a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:55.669759 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e30f1-c28d-4587-bfa7-40645b8a446a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:55.670041 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e30f1-c28d-4587-bfa7-40645b8a446a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:55.670874 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e30f1-c28d-4587-bfa7-40645b8a446a None admin] RBAC: Adding query filter params (interface=public, service_id=1d328e40d4014194a5f5d5dc59bd5285, region_id=RegionOne) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:55.671313 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e30f1-c28d-4587-bfa7-40645b8a446a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:55.672697 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e30f1-c28d-4587-bfa7-40645b8a446a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:55.685806 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 514/1004] 10.222.0.18 () {56 vars in 1251 bytes} [Mon Jul 25 19:09:55 2022] GET /identity/v3/endpoints?service_id=1d328e40d4014194a5f5d5dc59bd5285&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:55.687121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e30f1-c28d-4587-bfa7-40645b8a446a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:55.755267 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:55.813149 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df81c22f-4fd6-4ccb-9a5b-db7a5174e8ef None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:55.839352 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e30f1-c28d-4587-bfa7-40645b8a446a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:55.874708 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a10a85-f57c-4e52-8edd-cfe28f80d3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:55.875903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a10a85-f57c-4e52-8edd-cfe28f80d3a8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:55.876423 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a10a85-f57c-4e52-8edd-cfe28f80d3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:55.876910 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a10a85-f57c-4e52-8edd-cfe28f80d3a8 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:55.878037 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a10a85-f57c-4e52-8edd-cfe28f80d3a8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:55.878467 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a10a85-f57c-4e52-8edd-cfe28f80d3a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:55.880491 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a10a85-f57c-4e52-8edd-cfe28f80d3a8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:55.886862 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a7f65d-0b9d-4691-b746-8d260bf568ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:55.887582 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7f65d-0b9d-4691-b746-8d260bf568ea None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:55.887876 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7f65d-0b9d-4691-b746-8d260bf568ea None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:55.888150 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7f65d-0b9d-4691-b746-8d260bf568ea None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:55.894810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 491/1005] 10.222.0.18 () {56 vars in 1122 bytes} [Mon Jul 25 19:09:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:55.895015 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a7f65d-0b9d-4691-b746-8d260bf568ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:55.895015 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a7f65d-0b9d-4691-b746-8d260bf568ea None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:55.897165 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a10a85-f57c-4e52-8edd-cfe28f80d3a8 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:55.899915 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 515/1006] 10.222.0.18 () {56 vars in 1087 bytes} [Mon Jul 25 19:09:55 2022] GET /identity/v3/services => generated 1727 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:09:56.110712 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a10a85-f57c-4e52-8edd-cfe28f80d3a8 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:56.147723 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:56.160621 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f00510-ec7b-4d45-90b9-83e4896e6c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:56.161324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f00510-ec7b-4d45-90b9-83e4896e6c82 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:56.161615 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f00510-ec7b-4d45-90b9-83e4896e6c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:56.161896 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f00510-ec7b-4d45-90b9-83e4896e6c82 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:56.162691 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f00510-ec7b-4d45-90b9-83e4896e6c82 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:56.166977 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f00510-ec7b-4d45-90b9-83e4896e6c82 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:56.188256 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 492/1007] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:09:55 2022] POST /identity/v3/endpoints => generated 325 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:09:58.032248 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9b9b9-6053-4219-b653-414a75a3e894 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:58.033058 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9b9b9-6053-4219-b653-414a75a3e894 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:58.033244 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9b9b9-6053-4219-b653-414a75a3e894 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:58.035917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 516/1008] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:58.046300 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab57b4f9-896b-4a37-ac0e-60dfa01ec9e9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:58.047216 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab57b4f9-896b-4a37-ac0e-60dfa01ec9e9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:58.047356 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab57b4f9-896b-4a37-ac0e-60dfa01ec9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:58.076550 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ab57b4f9-896b-4a37-ac0e-60dfa01ec9e9 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:58.077842 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ab57b4f9-896b-4a37-ac0e-60dfa01ec9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:58.188796 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 493/1009] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:58 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:09:58.212384 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d9b9b9-6053-4219-b653-414a75a3e894 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:58.250960 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:58.291208 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b59a17-6c76-4569-b6db-8ec181225cb0 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:58.291680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b59a17-6c76-4569-b6db-8ec181225cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:58.292167 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b59a17-6c76-4569-b6db-8ec181225cb0 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:58.294958 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 494/1010] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:09:58.303622 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a32de-9867-459c-adc9-52239458a333 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:58.304115 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a32de-9867-459c-adc9-52239458a333 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:58.304597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a32de-9867-459c-adc9-52239458a333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:58.333781 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4b0a32de-9867-459c-adc9-52239458a333 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:58.339224 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4b0a32de-9867-459c-adc9-52239458a333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:58.401702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d9b9b9-6053-4219-b653-414a75a3e894 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:58.450094 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac2406-be90-4211-9e02-6a2a7bb17138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:58.451188 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac2406-be90-4211-9e02-6a2a7bb17138 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:58.451486 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac2406-be90-4211-9e02-6a2a7bb17138 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:58.451818 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac2406-be90-4211-9e02-6a2a7bb17138 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:58.452647 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac2406-be90-4211-9e02-6a2a7bb17138 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:58.454178 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac2406-be90-4211-9e02-6a2a7bb17138 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:58.479162 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 517/1011] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Jul 25 19:09:58 2022] POST /identity/v3/services => generated 242 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:09:58.507670 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 495/1012] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:58 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:09:58.531620 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac2406-be90-4211-9e02-6a2a7bb17138 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:58.723799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:58.757568 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac2406-be90-4211-9e02-6a2a7bb17138 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:58.828118 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d24984-690d-4527-804d-bf2874f024d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:58.829523 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d24984-690d-4527-804d-bf2874f024d5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:58.830026 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d24984-690d-4527-804d-bf2874f024d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:58.830503 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d24984-690d-4527-804d-bf2874f024d5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:58.831803 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d24984-690d-4527-804d-bf2874f024d5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:58.834369 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d24984-690d-4527-804d-bf2874f024d5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:58.859163 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-a3d24984-690d-4527-804d-bf2874f024d5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 19:09:58.859163 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:58.859163 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 19:09:58.859163 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 19:09:58.859163 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 19:09:58.859163 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 19:09:58.859163 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 19:09:58.859163 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:58.859163 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 19:09:58.859163 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:58.859163 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 19:09:58.859163 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:58.859163 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:09:58.859163 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:09:58.859163 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:09:58.859163 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:09:58.859163 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:09:58.861174 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:09:58.861174 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:09:58.861174 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:09:58.861174 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:09:58.861174 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:09:58.861174 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:09:58.861174 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 19:09:58.861174 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 19:09:58.861174 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 19:09:58.861174 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 19:09:58.861174 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:09:58.861174 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:09:58.861174 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:09:58.861174 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:09:58.861174 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:09:58.861174 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:09:58.862856 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:09:58.862856 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:09:58.862856 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:09:58.862856 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:09:58.862856 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:09:58.862856 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:09:58.862856 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:09:58.862856 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:09:58.862856 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:09:58.862856 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:09:58.862856 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 19:09:58.862856 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:09:58.862856 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 19:09:58.862856 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:09:58.862856 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 518/1013] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:09:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:09:58.872487 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0a32de-9867-459c-adc9-52239458a333 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:58.959475 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607318ba-d17f-433c-8ff6-293f79313aff None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:58.959779 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607318ba-d17f-433c-8ff6-293f79313aff None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:58.960131 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607318ba-d17f-433c-8ff6-293f79313aff None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:58.961740 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 519/1014] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:09:58.968283 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:58.968966 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05270362-a89f-489e-8bd4-6376aa38fc8d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:58.969249 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05270362-a89f-489e-8bd4-6376aa38fc8d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:58.969574 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05270362-a89f-489e-8bd4-6376aa38fc8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:58.994571 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-05270362-a89f-489e-8bd4-6376aa38fc8d None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:09:58.995955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-05270362-a89f-489e-8bd4-6376aa38fc8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:09:59.094242 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0a32de-9867-459c-adc9-52239458a333 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:59.103266 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 520/1015] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:09:58 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:09:59.144478 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ed695-357f-4e51-bf4d-1a61933ca8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:59.145605 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ed695-357f-4e51-bf4d-1a61933ca8a5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:59.145739 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05270362-a89f-489e-8bd4-6376aa38fc8d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:59.146226 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ed695-357f-4e51-bf4d-1a61933ca8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:59.146719 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ed695-357f-4e51-bf4d-1a61933ca8a5 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:59.147663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ed695-357f-4e51-bf4d-1a61933ca8a5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:59.148139 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ed695-357f-4e51-bf4d-1a61933ca8a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:59.150182 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ed695-357f-4e51-bf4d-1a61933ca8a5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:59.158062 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 496/1016] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Jul 25 19:09:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:59.171927 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ed695-357f-4e51-bf4d-1a61933ca8a5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:59.318860 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:59.347943 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ed695-357f-4e51-bf4d-1a61933ca8a5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:59.350571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:59.363441 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05270362-a89f-489e-8bd4-6376aa38fc8d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:59.406396 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc824f-1bef-4d60-9791-763a3dbba9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:59.407257 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc824f-1bef-4d60-9791-763a3dbba9b8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:59.407642 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc824f-1bef-4d60-9791-763a3dbba9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:59.407928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc824f-1bef-4d60-9791-763a3dbba9b8 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:59.409410 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fc824f-1bef-4d60-9791-763a3dbba9b8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:09:59.409717 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fc824f-1bef-4d60-9791-763a3dbba9b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:59.411655 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fc824f-1bef-4d60-9791-763a3dbba9b8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:59.419448 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 497/1017] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Jul 25 19:09:59 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:59.426010 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc824f-1bef-4d60-9791-763a3dbba9b8 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:59.437097 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b1db6-1d47-4f0b-a447-de985c5c3af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:59.438591 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b1db6-1d47-4f0b-a447-de985c5c3af0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:59.439070 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b1db6-1d47-4f0b-a447-de985c5c3af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:59.439575 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b1db6-1d47-4f0b-a447-de985c5c3af0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:59.441845 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b1db6-1d47-4f0b-a447-de985c5c3af0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:59.444681 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b1db6-1d47-4f0b-a447-de985c5c3af0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:59.448088 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 521/1018] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:09:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:09:59.457883 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b1db6-1d47-4f0b-a447-de985c5c3af0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:09:59.628819 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc824f-1bef-4d60-9791-763a3dbba9b8 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:59.651324 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b1db6-1d47-4f0b-a447-de985c5c3af0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:09:59.692677 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d46ab-e886-48d4-b884-2aa73de40c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:59.693674 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d46ab-e886-48d4-b884-2aa73de40c30 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:59.694072 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d46ab-e886-48d4-b884-2aa73de40c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:59.694480 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d46ab-e886-48d4-b884-2aa73de40c30 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:59.695409 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484d46ab-e886-48d4-b884-2aa73de40c30 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:59.698248 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484d46ab-e886-48d4-b884-2aa73de40c30 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:59.723763 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4610b502-2187-47c3-98cc-4f585463f26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:09:59.724914 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610b502-2187-47c3-98cc-4f585463f26a None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:09:59.725422 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610b502-2187-47c3-98cc-4f585463f26a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:09:59.725915 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610b502-2187-47c3-98cc-4f585463f26a None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:09:59.727406 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4610b502-2187-47c3-98cc-4f585463f26a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:09:59.739802 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 498/1019] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:09:59 2022] POST /identity/v3/endpoints => generated 334 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:09:59.740042 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4610b502-2187-47c3-98cc-4f585463f26a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:09:59.770499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:09:59.783106 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 522/1020] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Jul 25 19:09:59 2022] POST /identity/v3/projects => generated 311 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:10:00.936945 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8dca09-d0f5-49a8-a4a4-8588121c5cdb None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:00.937497 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8dca09-d0f5-49a8-a4a4-8588121c5cdb None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:00.937982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8dca09-d0f5-49a8-a4a4-8588121c5cdb None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:00.940671 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 499/1021] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:10:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:10:00.948967 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a76244-52db-40c8-ae1e-9e2610bfa3e0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:00.949811 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a76244-52db-40c8-ae1e-9e2610bfa3e0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:00.949811 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a76244-52db-40c8-ae1e-9e2610bfa3e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:00.974318 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e2a76244-52db-40c8-ae1e-9e2610bfa3e0 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:10:00.976469 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e2a76244-52db-40c8-ae1e-9e2610bfa3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:01.119394 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 523/1022] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:10:00 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 19:10:01.144308 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8dca09-d0f5-49a8-a4a4-8588121c5cdb None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:01.189801 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:01.361950 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8dca09-d0f5-49a8-a4a4-8588121c5cdb None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:01.440906 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce4fea-3c6d-4be0-8708-f554ca3429b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:01.442033 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce4fea-3c6d-4be0-8708-f554ca3429b7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:01.442489 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce4fea-3c6d-4be0-8708-f554ca3429b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:01.442911 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce4fea-3c6d-4be0-8708-f554ca3429b7 None admin] PATH_INFO: `/v3/services/image` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:01.443798 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce4fea-3c6d-4be0-8708-f554ca3429b7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:01.445872 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce4fea-3c6d-4be0-8708-f554ca3429b7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:01.460912 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-e3ce4fea-3c6d-4be0-8708-f554ca3429b7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 19:10:01.460912 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:10:01.460912 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 19:10:01.460912 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 19:10:01.460912 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 19:10:01.460912 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 19:10:01.460912 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 19:10:01.460912 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:10:01.460912 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 19:10:01.460912 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:10:01.460912 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 19:10:01.460912 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:10:01.460912 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:10:01.460912 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:10:01.460912 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:10:01.460912 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:10:01.460912 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:10:01.462918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:10:01.462918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:10:01.462918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:10:01.462918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:10:01.462918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:10:01.462918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:10:01.462918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 19:10:01.462918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 19:10:01.462918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 19:10:01.462918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 19:10:01.462918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:10:01.462918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:10:01.462918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:10:01.462918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:10:01.462918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:10:01.462918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:10:01.469394 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:10:01.469394 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:10:01.469394 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:10:01.469394 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:10:01.469394 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:10:01.469394 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:10:01.469394 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:10:01.469394 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:10:01.469394 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:10:01.469394 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:10:01.469394 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 19:10:01.469394 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:10:01.469394 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 19:10:01.469394 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:10:01.469394 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 500/1023] 10.222.0.18 () {56 vars in 1105 bytes} [Mon Jul 25 19:10:01 2022] GET /identity/v3/services/image => generated 86 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:10:01.474741 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a76244-52db-40c8-ae1e-9e2610bfa3e0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:01.657989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a76244-52db-40c8-ae1e-9e2610bfa3e0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:01.692144 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:01.704872 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8b78de-7ed1-4723-82c1-abc4e62f7d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:01.705737 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b78de-7ed1-4723-82c1-abc4e62f7d26 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:01.706151 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b78de-7ed1-4723-82c1-abc4e62f7d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:01.706558 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b78de-7ed1-4723-82c1-abc4e62f7d26 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:01.707423 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8b78de-7ed1-4723-82c1-abc4e62f7d26 None admin] RBAC: Adding query filter params (name=image) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:01.707896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8b78de-7ed1-4723-82c1-abc4e62f7d26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:01.709435 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8b78de-7ed1-4723-82c1-abc4e62f7d26 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:01.719793 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 524/1024] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:10:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:01.723427 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce4fea-3c6d-4be0-8708-f554ca3429b7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:01.845939 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:01.870441 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce4fea-3c6d-4be0-8708-f554ca3429b7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:01.922126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5d73d-3a02-4baf-90d6-0217c0ec68de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:01.923033 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5d73d-3a02-4baf-90d6-0217c0ec68de None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:01.923444 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5d73d-3a02-4baf-90d6-0217c0ec68de None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:01.923903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5d73d-3a02-4baf-90d6-0217c0ec68de None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:01.924758 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e5d73d-3a02-4baf-90d6-0217c0ec68de None admin] RBAC: Adding query filter params (type=image) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:01.925189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e5d73d-3a02-4baf-90d6-0217c0ec68de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:01.926860 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e5d73d-3a02-4baf-90d6-0217c0ec68de None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:01.939543 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 501/1025] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:10:01 2022] GET /identity/v3/services?type=image => generated 350 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:10:01.941381 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8b78de-7ed1-4723-82c1-abc4e62f7d26 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:02.158812 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8b78de-7ed1-4723-82c1-abc4e62f7d26 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:02.186536 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e7fbdd-cc9e-4060-aeb3-adf1aa3fec29 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:02.187451 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e7fbdd-cc9e-4060-aeb3-adf1aa3fec29 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:02.188261 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e7fbdd-cc9e-4060-aeb3-adf1aa3fec29 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:02.190758 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 502/1026] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:10:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:10:02.199933 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316ab61-b394-4cca-8664-43d1275fd4fa None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:02.200274 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316ab61-b394-4cca-8664-43d1275fd4fa None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:02.200451 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316ab61-b394-4cca-8664-43d1275fd4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:02.224657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ba44c-540d-49e4-ae93-f8457f79bf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:02.225800 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ba44c-540d-49e4-ae93-f8457f79bf58 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:02.226087 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ba44c-540d-49e4-ae93-f8457f79bf58 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:02.226363 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ba44c-540d-49e4-ae93-f8457f79bf58 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:02.227769 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284ba44c-540d-49e4-ae93-f8457f79bf58 None admin] RBAC: Adding query filter params (interface=public, service_id=e953db2c3bb9422f93e73f878b9a0dbb, region_id=RegionOne) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:02.228702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284ba44c-540d-49e4-ae93-f8457f79bf58 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:02.230649 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284ba44c-540d-49e4-ae93-f8457f79bf58 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:02.237449 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 525/1027] 10.222.0.18 () {56 vars in 1251 bytes} [Mon Jul 25 19:10:01 2022] GET /identity/v3/endpoints?service_id=e953db2c3bb9422f93e73f878b9a0dbb&interface=public®ion_id=RegionOne => generated 194 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:02.237898 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a316ab61-b394-4cca-8664-43d1275fd4fa None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:10:02.240201 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a316ab61-b394-4cca-8664-43d1275fd4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:02.245681 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ba44c-540d-49e4-ae93-f8457f79bf58 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:02.355062 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:02.370497 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 503/1028] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:10:02 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 19:10:02.413787 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a316ab61-b394-4cca-8664-43d1275fd4fa None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:02.430061 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ba44c-540d-49e4-ae93-f8457f79bf58 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:02.501905 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b1cde8-6a93-4fd8-8114-41cc77e87ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:02.503054 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1cde8-6a93-4fd8-8114-41cc77e87ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:02.503544 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1cde8-6a93-4fd8-8114-41cc77e87ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:02.504057 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1cde8-6a93-4fd8-8114-41cc77e87ad7 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:02.505327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b1cde8-6a93-4fd8-8114-41cc77e87ad7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:02.507927 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b1cde8-6a93-4fd8-8114-41cc77e87ad7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:02.522801 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 526/1029] 10.222.0.18 () {56 vars in 1087 bytes} [Mon Jul 25 19:10:02 2022] GET /identity/v3/services => generated 1957 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:10:02.576830 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:02.621729 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a316ab61-b394-4cca-8664-43d1275fd4fa None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:02.641799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d73666-244f-45f6-bb19-8e6b95938692 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:02.642273 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d73666-244f-45f6-bb19-8e6b95938692 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:02.642748 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d73666-244f-45f6-bb19-8e6b95938692 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:02.645338 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 527/1030] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:10:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:10:02.655617 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c45e23-a656-4a6a-88bc-cf7a25984021 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:02.656106 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c45e23-a656-4a6a-88bc-cf7a25984021 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:02.656605 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c45e23-a656-4a6a-88bc-cf7a25984021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:02.682090 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba25a94-31bd-437e-ac67-1709b7be99a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:02.683831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba25a94-31bd-437e-ac67-1709b7be99a0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:02.684362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba25a94-31bd-437e-ac67-1709b7be99a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:02.684904 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba25a94-31bd-437e-ac67-1709b7be99a0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:02.688739 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-34c45e23-a656-4a6a-88bc-cf7a25984021 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:10:02.690945 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-34c45e23-a656-4a6a-88bc-cf7a25984021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:02.705036 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba25a94-31bd-437e-ac67-1709b7be99a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:10:02.722890 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-fba25a94-31bd-437e-ac67-1709b7be99a0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 19:10:02.722890 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:10:02.722890 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:10:02.722890 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:10:02.722890 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:10:02.722890 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:10:02.722890 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:10:02.722890 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:10:02.722890 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:10:02.722890 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:10:02.722890 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:10:02.722890 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:10:02.722890 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:10:02.722890 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:10:02.722890 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:10:02.722890 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:10:02.725789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:10:02.725789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:10:02.725789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:10:02.725789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:10:02.725789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:10:02.725789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:10:02.725789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:10:02.725789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:10:02.725789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:10:02.725789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:10:02.725789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:10:02.725789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:10:02.725789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:10:02.725789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:10:02.725789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:10:02.725789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:10:02.727696 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:10:02.727696 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:10:02.727696 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:10:02.727696 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:10:02.727696 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:10:02.727696 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:10:02.727696 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:10:02.727696 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:10:02.727696 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:10:02.727696 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:10:02.727696 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 19:10:02.727696 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:10:02.727696 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 504/1031] 10.222.0.18 () {56 vars in 1120 bytes} [Mon Jul 25 19:10:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:10:02.733282 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba25a94-31bd-437e-ac67-1709b7be99a0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:02.834096 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 528/1032] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:10:02 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 19:10:02.879888 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c45e23-a656-4a6a-88bc-cf7a25984021 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:02.925979 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba25a94-31bd-437e-ac67-1709b7be99a0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:02.956896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:02.997423 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18849f-5c72-41f8-92b6-670820fe5d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:02.998302 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18849f-5c72-41f8-92b6-670820fe5d62 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:02.998847 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18849f-5c72-41f8-92b6-670820fe5d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:02.999009 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18849f-5c72-41f8-92b6-670820fe5d62 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:03.001382 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef18849f-5c72-41f8-92b6-670820fe5d62 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:03.001382 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef18849f-5c72-41f8-92b6-670820fe5d62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:03.002408 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef18849f-5c72-41f8-92b6-670820fe5d62 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:03.019548 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 505/1033] 10.222.0.18 () {56 vars in 1115 bytes} [Mon Jul 25 19:10:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:10:03.028704 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18849f-5c72-41f8-92b6-670820fe5d62 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:03.033971 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:03.063879 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c45e23-a656-4a6a-88bc-cf7a25984021 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:03.131920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b94e87-66de-4c08-aca9-bfb9098bb11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:03.133242 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b94e87-66de-4c08-aca9-bfb9098bb11e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:03.133727 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b94e87-66de-4c08-aca9-bfb9098bb11e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:03.133907 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b94e87-66de-4c08-aca9-bfb9098bb11e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:03.136491 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b94e87-66de-4c08-aca9-bfb9098bb11e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:03.138175 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b94e87-66de-4c08-aca9-bfb9098bb11e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:03.140703 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 529/1034] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:10:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:03.149321 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b94e87-66de-4c08-aca9-bfb9098bb11e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:03.214181 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18849f-5c72-41f8-92b6-670820fe5d62 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:03.284605 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8491419-8152-463d-a99b-5be97c004d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:03.285478 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8491419-8152-463d-a99b-5be97c004d4c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:03.285784 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8491419-8152-463d-a99b-5be97c004d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:03.286180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8491419-8152-463d-a99b-5be97c004d4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:03.303312 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8491419-8152-463d-a99b-5be97c004d4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:03.306811 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8491419-8152-463d-a99b-5be97c004d4c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:03.334355 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-c8491419-8152-463d-a99b-5be97c004d4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:10:03.334355 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:10:03.334355 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:10:03.334355 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:10:03.334355 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:10:03.334355 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:10:03.334355 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:10:03.334355 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:10:03.334355 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:10:03.334355 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:10:03.334355 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:10:03.334355 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:10:03.334355 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:10:03.334355 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:10:03.334355 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:10:03.336514 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:10:03.336514 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:10:03.336514 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:10:03.336514 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:10:03.336514 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:10:03.336514 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:10:03.336514 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:10:03.336514 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:10:03.336514 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:10:03.336514 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:10:03.336514 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:10:03.336514 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:10:03.336514 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:10:03.336514 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:10:03.336514 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:10:03.336514 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:10:03.336514 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:10:03.339672 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:10:03.339672 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:10:03.339672 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:10:03.339672 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:10:03.339672 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 506/1035] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:10:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:10:03.346252 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8491419-8152-463d-a99b-5be97c004d4c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:03.347115 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b94e87-66de-4c08-aca9-bfb9098bb11e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:03.371394 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:03.409260 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a5cdd-8c54-4d9e-9e34-b9d965fe91c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:03.410457 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a5cdd-8c54-4d9e-9e34-b9d965fe91c7 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:03.411369 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a5cdd-8c54-4d9e-9e34-b9d965fe91c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:03.411623 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a5cdd-8c54-4d9e-9e34-b9d965fe91c7 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:03.413010 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078a5cdd-8c54-4d9e-9e34-b9d965fe91c7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:03.416644 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078a5cdd-8c54-4d9e-9e34-b9d965fe91c7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:03.467079 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 530/1036] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Jul 25 19:10:03 2022] POST /identity/v3/users => generated 285 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:10:03.492069 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:03.589825 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8491419-8152-463d-a99b-5be97c004d4c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:03.637609 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7f0b9-cd58-4f94-8f2e-af58541f0b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:03.638504 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7f0b9-cd58-4f94-8f2e-af58541f0b4b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:03.639010 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7f0b9-cd58-4f94-8f2e-af58541f0b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:03.639475 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7f0b9-cd58-4f94-8f2e-af58541f0b4b None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:03.640562 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad7f0b9-cd58-4f94-8f2e-af58541f0b4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:03.641022 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad7f0b9-cd58-4f94-8f2e-af58541f0b4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:03.642686 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad7f0b9-cd58-4f94-8f2e-af58541f0b4b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:03.660301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 507/1037] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:10:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:10:03.668132 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a5cdd-8c54-4d9e-9e34-b9d965fe91c7 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:03.871916 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a5cdd-8c54-4d9e-9e34-b9d965fe91c7 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:03.944770 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b630c-6bb0-4955-bfc2-42e81fc4054f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:03.946476 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b630c-6bb0-4955-bfc2-42e81fc4054f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:03.947054 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b630c-6bb0-4955-bfc2-42e81fc4054f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:03.947835 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b630c-6bb0-4955-bfc2-42e81fc4054f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:03.965345 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b630c-6bb0-4955-bfc2-42e81fc4054f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:03.968625 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b630c-6bb0-4955-bfc2-42e81fc4054f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:03.998632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-0e2b630c-6bb0-4955-bfc2-42e81fc4054f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 19:10:03.998632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:10:03.998632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:10:03.998632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:10:03.998632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:10:03.998632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:10:03.998632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:10:03.998632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:10:03.998632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:10:03.998632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:10:03.998632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:10:03.998632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:10:03.998632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:10:03.998632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:10:03.998632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:10:03.998632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:10:04.001106 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:10:04.001106 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:10:04.001106 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:10:04.001106 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:10:04.001106 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:10:04.001106 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:10:04.001106 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:10:04.001106 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:10:04.001106 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:10:04.001106 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:10:04.001106 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:10:04.001106 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:10:04.001106 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:10:04.001106 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:10:04.001106 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:10:04.001106 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:10:04.001106 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:10:04.001106 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:10:04.003181 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:10:04.003181 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:10:04.003181 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:10:04.003181 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:10:04.003181 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:10:04.003181 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:10:04.003181 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:10:04.003181 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:10:04.003181 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 19:10:04.003181 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:10:04.003181 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 531/1038] 10.222.0.18 () {56 vars in 1093 bytes} [Mon Jul 25 19:10:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:10:04.007416 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7f0b9-cd58-4f94-8f2e-af58541f0b4b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:04.151842 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7f0b9-cd58-4f94-8f2e-af58541f0b4b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:04.180258 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:04.223296 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b2b0dd-7628-45c3-91b5-4b0f1862e7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:04.224532 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2b0dd-7628-45c3-91b5-4b0f1862e7b2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:04.225042 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2b0dd-7628-45c3-91b5-4b0f1862e7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:04.225538 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2b0dd-7628-45c3-91b5-4b0f1862e7b2 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:04.227217 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b2b0dd-7628-45c3-91b5-4b0f1862e7b2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:04.227815 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b2b0dd-7628-45c3-91b5-4b0f1862e7b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:04.231777 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b2b0dd-7628-45c3-91b5-4b0f1862e7b2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:04.274158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 508/1039] 10.222.0.18 () {56 vars in 1133 bytes} [Mon Jul 25 19:10:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:04.277550 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b630c-6bb0-4955-bfc2-42e81fc4054f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:04.345965 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:04.498467 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b630c-6bb0-4955-bfc2-42e81fc4054f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:04.570097 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f679c5a-06c5-4b93-8c28-3e8e0a1b670f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:04.571206 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f679c5a-06c5-4b93-8c28-3e8e0a1b670f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:04.572679 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f679c5a-06c5-4b93-8c28-3e8e0a1b670f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:04.573182 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f679c5a-06c5-4b93-8c28-3e8e0a1b670f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:04.715913 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f679c5a-06c5-4b93-8c28-3e8e0a1b670f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:04.719291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f679c5a-06c5-4b93-8c28-3e8e0a1b670f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:04.781760 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:04.927270 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 532/1040] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:10:04 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 648 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 19:10:04.928748 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b2b0dd-7628-45c3-91b5-4b0f1862e7b2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:05.074701 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b2b0dd-7628-45c3-91b5-4b0f1862e7b2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:05.127552 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99791265-9bc7-4837-bc29-f0233e1c8ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:05.128555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99791265-9bc7-4837-bc29-f0233e1c8ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:05.128979 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99791265-9bc7-4837-bc29-f0233e1c8ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:05.129404 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99791265-9bc7-4837-bc29-f0233e1c8ce1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:05.144363 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99791265-9bc7-4837-bc29-f0233e1c8ce1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:05.145858 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99791265-9bc7-4837-bc29-f0233e1c8ce1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:05.176741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-99791265-9bc7-4837-bc29-f0233e1c8ce1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:10:05.176741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:10:05.176741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:10:05.176741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:10:05.176741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:10:05.176741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:10:05.176741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:10:05.176741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:10:05.176741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:10:05.176741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:10:05.176741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:10:05.176741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:10:05.176741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:10:05.176741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:10:05.176741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:10:05.178539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:10:05.178539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:10:05.178539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:10:05.178539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:10:05.178539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:10:05.178539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:10:05.178539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:10:05.178539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:10:05.178539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:10:05.178539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:10:05.178539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:10:05.178539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:10:05.178539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:10:05.178539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:10:05.178539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:10:05.178539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:10:05.178539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:10:05.182255 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:10:05.182255 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:10:05.182255 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:10:05.182255 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:10:05.183919 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 509/1041] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:10:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:10:05.185168 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f679c5a-06c5-4b93-8c28-3e8e0a1b670f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:05.328682 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f679c5a-06c5-4b93-8c28-3e8e0a1b670f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:05.387204 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380a346-5e2d-48ea-96a7-59373499bfca None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:05.387646 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380a346-5e2d-48ea-96a7-59373499bfca None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:05.387967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380a346-5e2d-48ea-96a7-59373499bfca None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:05.389504 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 510/1042] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:10:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:10:05.389977 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:05.397661 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9c788-8f1b-427c-bb4e-c47946f4b326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:05.398166 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe4189-2e5f-4dc5-a6d9-071e30f8d3c8 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:05.398682 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe4189-2e5f-4dc5-a6d9-071e30f8d3c8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:05.398811 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9c788-8f1b-427c-bb4e-c47946f4b326 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:05.399250 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9c788-8f1b-427c-bb4e-c47946f4b326 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:05.399498 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe4189-2e5f-4dc5-a6d9-071e30f8d3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:05.399818 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9c788-8f1b-427c-bb4e-c47946f4b326 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:05.401221 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b9c788-8f1b-427c-bb4e-c47946f4b326 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:05.401747 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b9c788-8f1b-427c-bb4e-c47946f4b326 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:05.404356 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b9c788-8f1b-427c-bb4e-c47946f4b326 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:05.423741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-38fe4189-2e5f-4dc5-a6d9-071e30f8d3c8 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:10:05.425281 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-38fe4189-2e5f-4dc5-a6d9-071e30f8d3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:05.435883 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 533/1043] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:10:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:05.439635 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9c788-8f1b-427c-bb4e-c47946f4b326 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:05.470384 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:05.550785 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 511/1044] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:10:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 19:10:05.577099 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe4189-2e5f-4dc5-a6d9-071e30f8d3c8 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:05.651630 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9c788-8f1b-427c-bb4e-c47946f4b326 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:05.720592 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef33d2-3dcd-485b-8281-f4a893c7c616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:05.721978 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef33d2-3dcd-485b-8281-f4a893c7c616 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:05.722474 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef33d2-3dcd-485b-8281-f4a893c7c616 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:05.722973 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef33d2-3dcd-485b-8281-f4a893c7c616 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:05.733116 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe4189-2e5f-4dc5-a6d9-071e30f8d3c8 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:05.738891 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef33d2-3dcd-485b-8281-f4a893c7c616 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:05.741494 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef33d2-3dcd-485b-8281-f4a893c7c616 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:05.768757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-4fef33d2-3dcd-485b-8281-f4a893c7c616 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:10:05.768757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:10:05.768757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:10:05.768757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:10:05.768757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:10:05.768757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:10:05.768757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:10:05.768757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:10:05.768757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:10:05.768757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:10:05.768757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:10:05.768757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:10:05.768757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:10:05.768757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:10:05.768757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:10:05.770566 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:10:05.770566 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:10:05.770566 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:10:05.770566 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:10:05.770566 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:10:05.770566 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:10:05.770566 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:10:05.770566 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:10:05.770566 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:10:05.770566 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:10:05.770566 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:10:05.770566 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:10:05.770566 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:10:05.770566 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:10:05.770566 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:10:05.770566 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:10:05.770566 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:10:05.772387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:10:05.772387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:10:05.772387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:10:05.772387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:10:05.772387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:10:05.772387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:10:05.772387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:10:05.772387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:10:05.776299 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 534/1045] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:10:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:10:05.779083 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef33d2-3dcd-485b-8281-f4a893c7c616 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:05.806605 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587392a9-5939-4624-a584-2395186ee0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:05.808338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587392a9-5939-4624-a584-2395186ee0cf None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:05.808850 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587392a9-5939-4624-a584-2395186ee0cf None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:05.809369 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587392a9-5939-4624-a584-2395186ee0cf None admin] PATH_INFO: `/v3/services/image` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:05.810689 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587392a9-5939-4624-a584-2395186ee0cf None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:05.813344 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587392a9-5939-4624-a584-2395186ee0cf None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:05.836915 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-587392a9-5939-4624-a584-2395186ee0cf None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 19:10:05.836915 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:10:05.836915 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 19:10:05.836915 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 19:10:05.836915 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 19:10:05.836915 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 19:10:05.836915 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 19:10:05.836915 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:10:05.836915 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 19:10:05.836915 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:10:05.836915 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 19:10:05.836915 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:10:05.836915 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:10:05.836915 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:10:05.836915 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:10:05.836915 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:10:05.836915 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:10:05.840248 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:10:05.840248 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:10:05.840248 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:10:05.840248 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:10:05.840248 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:10:05.840248 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:10:05.840248 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 19:10:05.840248 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 19:10:05.840248 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 19:10:05.840248 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 19:10:05.840248 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:10:05.840248 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:10:05.840248 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:10:05.840248 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:10:05.840248 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:10:05.840248 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:10:05.842582 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:10:05.842582 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:10:05.842582 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:10:05.842582 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:10:05.842582 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:10:05.842582 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:10:05.842582 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:10:05.842582 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:10:05.842582 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:10:05.842582 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:10:05.842582 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 19:10:05.842582 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:10:05.842582 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 19:10:05.842582 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:10:05.847218 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 512/1046] 10.222.0.18 () {56 vars in 1105 bytes} [Mon Jul 25 19:10:05 2022] GET /identity/v3/services/image => generated 86 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:10:05.849118 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587392a9-5939-4624-a584-2395186ee0cf None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:05.863471 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:05.893210 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:05.935637 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef33d2-3dcd-485b-8281-f4a893c7c616 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:06.004827 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c887264-a0db-41df-ad4d-1121b5c1313b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:06.005559 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c887264-a0db-41df-ad4d-1121b5c1313b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:06.006137 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c887264-a0db-41df-ad4d-1121b5c1313b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:06.006575 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c887264-a0db-41df-ad4d-1121b5c1313b None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:06.008160 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c887264-a0db-41df-ad4d-1121b5c1313b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:06.008728 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c887264-a0db-41df-ad4d-1121b5c1313b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:06.012602 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c887264-a0db-41df-ad4d-1121b5c1313b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:06.044941 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 535/1047] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Jul 25 19:10:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:10:06.049585 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c887264-a0db-41df-ad4d-1121b5c1313b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:06.051833 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587392a9-5939-4624-a584-2395186ee0cf None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:06.102784 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a97d9-b3e8-419f-a0b8-66889fe1d517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:06.104509 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a97d9-b3e8-419f-a0b8-66889fe1d517 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:06.104509 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a97d9-b3e8-419f-a0b8-66889fe1d517 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:06.104509 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a97d9-b3e8-419f-a0b8-66889fe1d517 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:06.105008 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a97d9-b3e8-419f-a0b8-66889fe1d517 None admin] RBAC: Adding query filter params (name=image) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:06.105321 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a97d9-b3e8-419f-a0b8-66889fe1d517 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:06.106941 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a97d9-b3e8-419f-a0b8-66889fe1d517 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:06.113709 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 513/1048] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:10:05 2022] GET /identity/v3/services?name=image => generated 122 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:06.120683 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a97d9-b3e8-419f-a0b8-66889fe1d517 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:06.255454 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c887264-a0db-41df-ad4d-1121b5c1313b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:06.269887 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:06.288093 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:06.300810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a97d9-b3e8-419f-a0b8-66889fe1d517 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:06.328627 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89df3b9-9a62-435f-bbbe-dd379ea00f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:06.329495 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89df3b9-9a62-435f-bbbe-dd379ea00f6a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:06.329888 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89df3b9-9a62-435f-bbbe-dd379ea00f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:06.330229 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89df3b9-9a62-435f-bbbe-dd379ea00f6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:06.331129 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89df3b9-9a62-435f-bbbe-dd379ea00f6a None admin] RBAC: Adding query filter params (role.id=e1dc00443f96426e9edf6ea6ca8ae542, scope.project.id=191069caccd04233b012585e29cfa18b, user.id=860e7d26ad364795bb25836aadd4ac7b) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:06.331577 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89df3b9-9a62-435f-bbbe-dd379ea00f6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:06.333098 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89df3b9-9a62-435f-bbbe-dd379ea00f6a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:06.341442 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 536/1049] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:10:06 2022] GET /identity/v3/role_assignments?user.id=860e7d26ad364795bb25836aadd4ac7b&scope.project.id=191069caccd04233b012585e29cfa18b&role.id=e1dc00443f96426e9edf6ea6ca8ae542 => generated 259 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:10:06.343086 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5e7e9-7ac7-4e48-bce9-f73a9b4e05f1 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:06.343475 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5e7e9-7ac7-4e48-bce9-f73a9b4e05f1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:06.343807 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5e7e9-7ac7-4e48-bce9-f73a9b4e05f1 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:06.345793 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 537/1050] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:10:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:10:06.351026 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70bfbfe-a2bf-4736-8d0f-6c70d739c3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:06.351941 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70bfbfe-a2bf-4736-8d0f-6c70d739c3f3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:06.352126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70bfbfe-a2bf-4736-8d0f-6c70d739c3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:06.352417 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70bfbfe-a2bf-4736-8d0f-6c70d739c3f3 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:06.352857 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7d6d3-c057-40e9-832b-aca7ed7414d0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:06.353835 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7d6d3-c057-40e9-832b-aca7ed7414d0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:06.354259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7d6d3-c057-40e9-832b-aca7ed7414d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:06.355233 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70bfbfe-a2bf-4736-8d0f-6c70d739c3f3 None admin] RBAC: Adding query filter params (type=image) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:06.355655 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70bfbfe-a2bf-4736-8d0f-6c70d739c3f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:06.357326 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70bfbfe-a2bf-4736-8d0f-6c70d739c3f3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:06.364698 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 514/1051] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:10:06 2022] GET /identity/v3/services?type=image => generated 350 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:10:06.372035 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70bfbfe-a2bf-4736-8d0f-6c70d739c3f3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:06.377719 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-80e7d6d3-c057-40e9-832b-aca7ed7414d0 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:10:06.379326 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-80e7d6d3-c057-40e9-832b-aca7ed7414d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:06.480220 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 538/1052] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:10:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 19:10:06.509024 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7d6d3-c057-40e9-832b-aca7ed7414d0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:06.534387 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70bfbfe-a2bf-4736-8d0f-6c70d739c3f3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:06.589714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74920ab-d13e-4890-a56e-cdfee7e2d190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:06.590426 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74920ab-d13e-4890-a56e-cdfee7e2d190 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:06.590706 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74920ab-d13e-4890-a56e-cdfee7e2d190 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:06.591379 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74920ab-d13e-4890-a56e-cdfee7e2d190 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:06.592332 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74920ab-d13e-4890-a56e-cdfee7e2d190 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:06.593803 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74920ab-d13e-4890-a56e-cdfee7e2d190 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:06.628877 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 515/1053] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:10:06 2022] POST /identity/v3/endpoints => generated 327 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:10:06.673702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7d6d3-c057-40e9-832b-aca7ed7414d0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:06.685316 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:06.737825 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d1074-dab2-40ff-9388-f69385643ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:06.739270 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d1074-dab2-40ff-9388-f69385643ec9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:06.739886 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d1074-dab2-40ff-9388-f69385643ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:06.740393 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d1074-dab2-40ff-9388-f69385643ec9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:06.756941 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d1074-dab2-40ff-9388-f69385643ec9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:10:06.766957 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e94d1074-dab2-40ff-9388-f69385643ec9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:10:06.766957 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:10:06.766957 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:10:06.766957 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:10:06.766957 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:10:06.766957 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:10:06.766957 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:10:06.766957 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:10:06.766957 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:10:06.766957 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:10:06.766957 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:10:06.766957 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:10:06.766957 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:10:06.766957 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:10:06.766957 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:10:06.766957 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:10:06.776808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:10:06.776808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:10:06.776808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:10:06.776808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:10:06.776808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:10:06.776808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:10:06.776808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:10:06.776808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:10:06.776808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:10:06.776808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:10:06.776808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:10:06.776808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:10:06.776808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:10:06.776808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:10:06.776808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:10:06.776808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:10:06.777880 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:10:06.777880 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:10:06.777880 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:10:06.777880 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:10:06.777880 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:10:06.777880 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:10:06.777880 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:10:06.777880 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:10:06.777880 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:10:06.777880 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:10:06.777880 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:10:06.777880 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:10:06.777880 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 539/1054] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Jul 25 19:10:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:10:06.781835 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74920ab-d13e-4890-a56e-cdfee7e2d190 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:06.785308 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:06.960532 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74920ab-d13e-4890-a56e-cdfee7e2d190 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:07.008967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc907a8-b836-4350-8c6a-bdb290726c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:07.010505 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc907a8-b836-4350-8c6a-bdb290726c77 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:07.010785 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc907a8-b836-4350-8c6a-bdb290726c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:07.011076 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc907a8-b836-4350-8c6a-bdb290726c77 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:07.011908 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc907a8-b836-4350-8c6a-bdb290726c77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:07.012215 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc907a8-b836-4350-8c6a-bdb290726c77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:07.013908 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc907a8-b836-4350-8c6a-bdb290726c77 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:07.026662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 516/1055] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:10:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:07.033953 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d1074-dab2-40ff-9388-f69385643ec9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:07.195432 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d1074-dab2-40ff-9388-f69385643ec9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:07.247040 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0988e7-94cb-4f9d-ac82-b84a15554744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:07.247982 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0988e7-94cb-4f9d-ac82-b84a15554744 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:07.248265 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0988e7-94cb-4f9d-ac82-b84a15554744 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:07.248575 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0988e7-94cb-4f9d-ac82-b84a15554744 None admin] PATH_INFO: `/v3/users/8bb376a751024bdf8ef1d9e156cfdef3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:07.270525 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0988e7-94cb-4f9d-ac82-b84a15554744 None admin] RBAC: Authorizing `identity:get_user(user_id=8bb376a751024bdf8ef1d9e156cfdef3)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:07.272730 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0988e7-94cb-4f9d-ac82-b84a15554744 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:07.276414 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 540/1056] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:10:07 2022] GET /identity/v3/users/8bb376a751024bdf8ef1d9e156cfdef3 => generated 285 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:07.283565 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc907a8-b836-4350-8c6a-bdb290726c77 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:07.388397 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:07.456290 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc907a8-b836-4350-8c6a-bdb290726c77 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:07.510075 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6692630-5d83-476f-ab3c-b0ccf026898f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:07.510899 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6692630-5d83-476f-ab3c-b0ccf026898f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:07.511194 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6692630-5d83-476f-ab3c-b0ccf026898f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:07.511470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6692630-5d83-476f-ab3c-b0ccf026898f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:07.614276 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6692630-5d83-476f-ab3c-b0ccf026898f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:07.616003 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6692630-5d83-476f-ab3c-b0ccf026898f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:07.718000 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:07.783574 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 517/1057] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:10:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 19:10:07.789278 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0988e7-94cb-4f9d-ac82-b84a15554744 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:07.825674 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:07.954221 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0988e7-94cb-4f9d-ac82-b84a15554744 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:08.001487 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b403cf-cf93-44ab-8296-dc7d35ae7f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:08.002378 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b403cf-cf93-44ab-8296-dc7d35ae7f4c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:08.002663 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b403cf-cf93-44ab-8296-dc7d35ae7f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:08.002933 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b403cf-cf93-44ab-8296-dc7d35ae7f4c None admin] PATH_INFO: `/v3/projects/2848e952fdde4cc885cce54b849bc549` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:08.004516 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b403cf-cf93-44ab-8296-dc7d35ae7f4c None admin] RBAC: Authorizing `identity:get_project(project_id=2848e952fdde4cc885cce54b849bc549)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:08.006023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b403cf-cf93-44ab-8296-dc7d35ae7f4c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:08.008561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 541/1058] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:10:07 2022] GET /identity/v3/projects/2848e952fdde4cc885cce54b849bc549 => generated 311 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:08.015959 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6692630-5d83-476f-ab3c-b0ccf026898f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:08.200338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6692630-5d83-476f-ab3c-b0ccf026898f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:08.249204 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab464a-59c7-4fad-ab37-f55a4f883eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:08.249793 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab464a-59c7-4fad-ab37-f55a4f883eae None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:08.250065 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab464a-59c7-4fad-ab37-f55a4f883eae None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:08.250515 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab464a-59c7-4fad-ab37-f55a4f883eae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:08.251300 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ab464a-59c7-4fad-ab37-f55a4f883eae None admin] RBAC: Adding query filter params (role.id=ed7fec18968241efa76a0ee769d96da9, scope.project.id=2848e952fdde4cc885cce54b849bc549, user.id=8bb376a751024bdf8ef1d9e156cfdef3) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:08.251658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ab464a-59c7-4fad-ab37-f55a4f883eae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:08.253215 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ab464a-59c7-4fad-ab37-f55a4f883eae None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:08.267998 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 518/1059] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:10:08 2022] GET /identity/v3/role_assignments?user.id=8bb376a751024bdf8ef1d9e156cfdef3&scope.project.id=2848e952fdde4cc885cce54b849bc549&role.id=ed7fec18968241efa76a0ee769d96da9 => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:09.413206 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12709d-3b69-4451-8601-da279d38e30e None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:09.413817 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12709d-3b69-4451-8601-da279d38e30e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:09.414193 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12709d-3b69-4451-8601-da279d38e30e None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:09.418796 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 542/1060] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:10:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:10:09.428035 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1d647-1c00-4ba2-8d53-85e332c2283c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:09.428611 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1d647-1c00-4ba2-8d53-85e332c2283c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:09.428905 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1d647-1c00-4ba2-8d53-85e332c2283c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:09.463879 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d4d1d647-1c00-4ba2-8d53-85e332c2283c None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:10:09.466523 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d4d1d647-1c00-4ba2-8d53-85e332c2283c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:09.519480 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:09.591574 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 519/1061] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:10:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:10:09.626167 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd12709d-3b69-4451-8601-da279d38e30e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:09.637694 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9caf10e-03ae-4b3e-901d-61eb80976999 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:09.638042 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9caf10e-03ae-4b3e-901d-61eb80976999 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:09.638262 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9caf10e-03ae-4b3e-901d-61eb80976999 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:09.639972 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 520/1062] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:10:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:10:09.646178 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f228f0f-9a8a-49c7-90c3-5351355b988e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:09.646823 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f228f0f-9a8a-49c7-90c3-5351355b988e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:09.647350 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f228f0f-9a8a-49c7-90c3-5351355b988e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:09.669010 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8f228f0f-9a8a-49c7-90c3-5351355b988e None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:10:09.670239 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8f228f0f-9a8a-49c7-90c3-5351355b988e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:09.787131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 521/1063] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:10:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 19:10:09.792025 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:09.817084 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd12709d-3b69-4451-8601-da279d38e30e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:09.828261 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f228f0f-9a8a-49c7-90c3-5351355b988e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:09.881421 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9730e7b-b4f3-4229-a928-c733a2d80a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:09.883002 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9730e7b-b4f3-4229-a928-c733a2d80a38 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:09.883547 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9730e7b-b4f3-4229-a928-c733a2d80a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:09.884674 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9730e7b-b4f3-4229-a928-c733a2d80a38 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:09.906188 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9730e7b-b4f3-4229-a928-c733a2d80a38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:10:09.918929 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b9730e7b-b4f3-4229-a928-c733a2d80a38 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 19:10:09.918929 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:10:09.918929 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:10:09.918929 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:10:09.918929 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:10:09.918929 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:10:09.918929 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:10:09.918929 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:10:09.918929 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:10:09.918929 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:10:09.918929 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:10:09.918929 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:10:09.918929 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:10:09.918929 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:10:09.918929 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:10:09.918929 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:10:09.920796 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:10:09.920796 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:10:09.920796 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:10:09.920796 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:10:09.920796 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:10:09.920796 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:10:09.920796 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:10:09.920796 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:10:09.920796 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:10:09.920796 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:10:09.920796 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:10:09.920796 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:10:09.920796 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:10:09.920796 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:10:09.920796 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:10:09.920796 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:10:09.922489 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:10:09.922489 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:10:09.922489 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:10:09.922489 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:10:09.922489 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:10:09.922489 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:10:09.922489 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:10:09.922489 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:10:09.922489 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:10:09.922489 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:10:09.922489 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 19:10:09.922489 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:10:09.922489 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 543/1064] 10.222.0.18 () {56 vars in 1120 bytes} [Mon Jul 25 19:10:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:10:09.928991 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9730e7b-b4f3-4229-a928-c733a2d80a38 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:09.983827 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:10.002848 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f228f0f-9a8a-49c7-90c3-5351355b988e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:10.069162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c78591-f4f5-451b-a85b-4bbddb31b1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:10.070325 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c78591-f4f5-451b-a85b-4bbddb31b1a0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:10.070842 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c78591-f4f5-451b-a85b-4bbddb31b1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:10.071349 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c78591-f4f5-451b-a85b-4bbddb31b1a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:10.111092 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9730e7b-b4f3-4229-a928-c733a2d80a38 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:10.182423 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f74c80-33f7-4d53-98d7-42440fa56853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:10.183541 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f74c80-33f7-4d53-98d7-42440fa56853 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:10.184044 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f74c80-33f7-4d53-98d7-42440fa56853 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:10.184524 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f74c80-33f7-4d53-98d7-42440fa56853 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:10.185876 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f74c80-33f7-4d53-98d7-42440fa56853 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:10.186403 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f74c80-33f7-4d53-98d7-42440fa56853 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:10.188995 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f74c80-33f7-4d53-98d7-42440fa56853 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:10.194491 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c78591-f4f5-451b-a85b-4bbddb31b1a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:10.196172 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c78591-f4f5-451b-a85b-4bbddb31b1a0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:10.213058 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 544/1065] 10.222.0.18 () {56 vars in 1115 bytes} [Mon Jul 25 19:10:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:10:10.216954 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f74c80-33f7-4d53-98d7-42440fa56853 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:10.246420 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:10.357865 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:10.387608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 522/1066] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:10:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 19:10:10.395446 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c78591-f4f5-451b-a85b-4bbddb31b1a0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:10.425633 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f74c80-33f7-4d53-98d7-42440fa56853 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:10.491868 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1438d-2080-4392-a990-318e08669ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:10.493265 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1438d-2080-4392-a990-318e08669ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:10.494314 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1438d-2080-4392-a990-318e08669ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:10.494995 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1438d-2080-4392-a990-318e08669ba3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:10.514475 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b1438d-2080-4392-a990-318e08669ba3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:10.516709 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b1438d-2080-4392-a990-318e08669ba3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:10.552922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-75b1438d-2080-4392-a990-318e08669ba3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:10:10.552922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:10:10.552922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:10:10.552922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:10:10.552922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:10:10.552922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:10:10.552922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:10:10.552922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:10:10.552922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:10:10.552922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:10:10.552922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:10:10.552922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:10:10.552922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:10:10.552922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:10:10.552922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:10:10.554678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:10:10.554678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:10:10.554678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:10:10.554678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:10:10.554678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:10:10.554678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:10:10.554678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:10:10.554678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:10:10.554678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:10:10.554678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:10:10.554678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:10:10.554678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:10:10.554678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:10:10.554678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:10:10.554678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:10:10.554678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:10:10.554678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:10:10.558897 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:10:10.558897 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:10:10.558897 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:10:10.558897 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:10:10.558897 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 545/1067] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:10:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:10:10.562026 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1438d-2080-4392-a990-318e08669ba3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:10.592262 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c78591-f4f5-451b-a85b-4bbddb31b1a0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:10.655076 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:10.659965 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa3a4b7-ee1b-48de-aea0-e2fbfcf8df0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:10.660797 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa3a4b7-ee1b-48de-aea0-e2fbfcf8df0d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:10.661075 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa3a4b7-ee1b-48de-aea0-e2fbfcf8df0d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:10.661354 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa3a4b7-ee1b-48de-aea0-e2fbfcf8df0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:10.676403 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa3a4b7-ee1b-48de-aea0-e2fbfcf8df0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:10.678452 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa3a4b7-ee1b-48de-aea0-e2fbfcf8df0d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:10.704244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-4fa3a4b7-ee1b-48de-aea0-e2fbfcf8df0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:10:10.704244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:10:10.704244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:10:10.704244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:10:10.704244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:10:10.704244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:10:10.704244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:10:10.704244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:10:10.704244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:10:10.704244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:10:10.704244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:10:10.704244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:10:10.704244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:10:10.704244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:10:10.704244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:10:10.713669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:10:10.713669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:10:10.713669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:10:10.713669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:10:10.713669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:10:10.713669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:10:10.713669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:10:10.713669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:10:10.713669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:10:10.713669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:10:10.713669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:10:10.713669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:10:10.713669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:10:10.713669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:10:10.713669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:10:10.713669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:10:10.713669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:10:10.715613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:10:10.715613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:10:10.715613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:10:10.715613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:10:10.715613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 523/1068] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:10:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:10:10.715613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa3a4b7-ee1b-48de-aea0-e2fbfcf8df0d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:10.745690 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1438d-2080-4392-a990-318e08669ba3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:10.786213 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:10.800568 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222ff6f7-f2c7-45db-a0c6-987446fd09c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:10.801744 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ff6f7-f2c7-45db-a0c6-987446fd09c5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:10.802264 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ff6f7-f2c7-45db-a0c6-987446fd09c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:10.802783 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ff6f7-f2c7-45db-a0c6-987446fd09c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:10.804233 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222ff6f7-f2c7-45db-a0c6-987446fd09c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:10.804855 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222ff6f7-f2c7-45db-a0c6-987446fd09c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:10.808133 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222ff6f7-f2c7-45db-a0c6-987446fd09c5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:10.832638 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 546/1069] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:10:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:10:10.841400 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222ff6f7-f2c7-45db-a0c6-987446fd09c5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:10.896352 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa3a4b7-ee1b-48de-aea0-e2fbfcf8df0d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:10.970792 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa2f21e-1906-4526-89a6-c77bafb1c343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:10.972439 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2f21e-1906-4526-89a6-c77bafb1c343 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:10.972949 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2f21e-1906-4526-89a6-c77bafb1c343 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:10.973465 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa2f21e-1906-4526-89a6-c77bafb1c343 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:10.974867 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2f21e-1906-4526-89a6-c77bafb1c343 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:10.975374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2f21e-1906-4526-89a6-c77bafb1c343 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:10.975706 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222ff6f7-f2c7-45db-a0c6-987446fd09c5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:10.978147 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa2f21e-1906-4526-89a6-c77bafb1c343 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:11.003096 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 524/1070] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:10:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:11.008879 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:11.035231 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8af0dd-b5b7-446b-ab66-b9fe9b58fb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:11.036664 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8af0dd-b5b7-446b-ab66-b9fe9b58fb71 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:11.037136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8af0dd-b5b7-446b-ab66-b9fe9b58fb71 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:11.037613 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8af0dd-b5b7-446b-ab66-b9fe9b58fb71 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:11.053170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8af0dd-b5b7-446b-ab66-b9fe9b58fb71 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:11.056193 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8af0dd-b5b7-446b-ab66-b9fe9b58fb71 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:11.087965 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-1a8af0dd-b5b7-446b-ab66-b9fe9b58fb71 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 19:10:11.087965 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:10:11.087965 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:10:11.087965 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:10:11.087965 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:10:11.087965 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:10:11.087965 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:10:11.087965 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:10:11.087965 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:10:11.087965 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:10:11.087965 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:10:11.087965 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:10:11.087965 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:10:11.087965 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:10:11.087965 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:10:11.087965 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:10:11.089824 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:10:11.089824 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:10:11.089824 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:10:11.089824 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:10:11.089824 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:10:11.089824 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:10:11.089824 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:10:11.089824 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:10:11.089824 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:10:11.089824 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:10:11.089824 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:10:11.089824 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:10:11.089824 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:10:11.089824 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:10:11.089824 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:10:11.089824 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:10:11.089824 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:10:11.089824 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:10:11.092871 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:10:11.092871 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:10:11.092871 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:10:11.092871 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:10:11.092871 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:10:11.092871 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:10:11.092871 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:10:11.092871 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:10:11.092871 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 19:10:11.092871 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:10:11.095197 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 547/1071] 10.222.0.18 () {56 vars in 1093 bytes} [Mon Jul 25 19:10:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:10:11.100975 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8af0dd-b5b7-446b-ab66-b9fe9b58fb71 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:11.101232 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece76c9-c383-40cc-85ea-6a2e84034176 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:11.101773 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece76c9-c383-40cc-85ea-6a2e84034176 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:11.102089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece76c9-c383-40cc-85ea-6a2e84034176 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:11.104514 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 525/1072] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:10:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:10:11.111704 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b11397-bcb9-4441-b41a-48733c53dea1 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:11.112135 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b11397-bcb9-4441-b41a-48733c53dea1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:11.112585 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b11397-bcb9-4441-b41a-48733c53dea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:11.139000 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b3b11397-bcb9-4441-b41a-48733c53dea1 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:10:11.141436 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b3b11397-bcb9-4441-b41a-48733c53dea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:11.286261 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8af0dd-b5b7-446b-ab66-b9fe9b58fb71 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:11.302368 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 526/1073] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:10:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 19:10:11.329839 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b11397-bcb9-4441-b41a-48733c53dea1 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:11.357954 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3969c57b-da24-4a04-b2f6-91eee2b78be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:11.358825 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3969c57b-da24-4a04-b2f6-91eee2b78be4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:11.359413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3969c57b-da24-4a04-b2f6-91eee2b78be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:11.359821 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3969c57b-da24-4a04-b2f6-91eee2b78be4 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:11.361163 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3969c57b-da24-4a04-b2f6-91eee2b78be4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:11.361783 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3969c57b-da24-4a04-b2f6-91eee2b78be4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:11.363471 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3969c57b-da24-4a04-b2f6-91eee2b78be4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:11.397734 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:11.399271 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 548/1074] 10.222.0.18 () {56 vars in 1133 bytes} [Mon Jul 25 19:10:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:10:11.409975 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3969c57b-da24-4a04-b2f6-91eee2b78be4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:11.483659 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:11.546215 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b11397-bcb9-4441-b41a-48733c53dea1 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:11.601392 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3969c57b-da24-4a04-b2f6-91eee2b78be4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:11.619052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b4b5d-923a-4270-a296-5d4da65ec255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:11.620330 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b4b5d-923a-4270-a296-5d4da65ec255 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:11.620661 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b4b5d-923a-4270-a296-5d4da65ec255 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:11.621067 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b4b5d-923a-4270-a296-5d4da65ec255 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:11.640620 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839b4b5d-923a-4270-a296-5d4da65ec255 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:10:11.656530 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1e41e-e9be-4830-8f53-24831ba814c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:11.656859 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-839b4b5d-923a-4270-a296-5d4da65ec255 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:10:11.656859 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:10:11.656859 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:10:11.656859 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:10:11.656859 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:10:11.656859 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:10:11.656859 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:10:11.656859 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:10:11.656859 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:10:11.656859 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:10:11.656859 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:10:11.656859 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:10:11.656859 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:10:11.656859 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:10:11.656859 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:10:11.656859 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:10:11.658189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:10:11.658189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:10:11.658189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:10:11.658189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:10:11.658189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:10:11.658189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:10:11.658189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:10:11.658189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:10:11.658189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:10:11.658189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:10:11.658189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:10:11.658189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:10:11.658189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:10:11.658189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:10:11.658189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:10:11.658189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:10:11.659159 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:10:11.659159 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:10:11.659159 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:10:11.659159 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:10:11.659159 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:10:11.659159 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:10:11.659159 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:10:11.659159 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:10:11.659159 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:10:11.659159 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:10:11.659159 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:10:11.659159 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:10:11.660011 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1e41e-e9be-4830-8f53-24831ba814c1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:11.660011 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1e41e-e9be-4830-8f53-24831ba814c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:11.660011 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1e41e-e9be-4830-8f53-24831ba814c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:11.660216 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 527/1075] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Jul 25 19:10:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:10:11.669878 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b4b5d-923a-4270-a296-5d4da65ec255 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:11.677150 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f1e41e-e9be-4830-8f53-24831ba814c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:11.678651 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f1e41e-e9be-4830-8f53-24831ba814c1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:11.699915 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-70f1e41e-e9be-4830-8f53-24831ba814c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:10:11.699915 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:10:11.699915 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:10:11.699915 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:10:11.699915 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:10:11.699915 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:10:11.699915 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:10:11.699915 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:10:11.699915 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:10:11.699915 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:10:11.699915 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:10:11.699915 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:10:11.699915 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:10:11.699915 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:10:11.699915 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:10:11.701133 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:10:11.701133 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:10:11.701133 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:10:11.701133 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:10:11.701133 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:10:11.701133 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:10:11.701133 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:10:11.701133 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:10:11.701133 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:10:11.701133 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:10:11.701133 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:10:11.701133 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:10:11.701133 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:10:11.701133 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:10:11.701133 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:10:11.701133 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:10:11.701133 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:10:11.702623 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:10:11.702623 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:10:11.702623 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:10:11.702623 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:10:11.702623 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 549/1076] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:10:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:10:11.707905 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1e41e-e9be-4830-8f53-24831ba814c1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:11.833045 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:11.838350 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:11.851403 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1e41e-e9be-4830-8f53-24831ba814c1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:11.854352 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b4b5d-923a-4270-a296-5d4da65ec255 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:11.901328 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758ce2c-3ded-4960-842e-2e3f58dc27ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:11.902015 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758ce2c-3ded-4960-842e-2e3f58dc27ea None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:11.902328 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758ce2c-3ded-4960-842e-2e3f58dc27ea None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:11.902704 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758ce2c-3ded-4960-842e-2e3f58dc27ea None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:11.903580 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7758ce2c-3ded-4960-842e-2e3f58dc27ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:11.903896 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7758ce2c-3ded-4960-842e-2e3f58dc27ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:11.905360 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7758ce2c-3ded-4960-842e-2e3f58dc27ea None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:11.917851 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 528/1077] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:10:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:10:11.918145 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c10fe-4a72-486c-a504-61b05040d0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:11.918576 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c10fe-4a72-486c-a504-61b05040d0ca None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:11.918921 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c10fe-4a72-486c-a504-61b05040d0ca None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:11.919258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c10fe-4a72-486c-a504-61b05040d0ca None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:11.921175 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994c10fe-4a72-486c-a504-61b05040d0ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:11.921175 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994c10fe-4a72-486c-a504-61b05040d0ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:11.922242 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994c10fe-4a72-486c-a504-61b05040d0ca None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:11.926653 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758ce2c-3ded-4960-842e-2e3f58dc27ea None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:11.942461 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 550/1078] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:10:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:11.952904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c10fe-4a72-486c-a504-61b05040d0ca None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:12.105853 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758ce2c-3ded-4960-842e-2e3f58dc27ea None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:12.142474 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c10fe-4a72-486c-a504-61b05040d0ca None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:12.158097 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f1c3b-2cfb-4a73-8d33-ef11b34a85c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:12.159058 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f1c3b-2cfb-4a73-8d33-ef11b34a85c5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:12.159365 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f1c3b-2cfb-4a73-8d33-ef11b34a85c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:12.159788 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f1c3b-2cfb-4a73-8d33-ef11b34a85c5 None admin] PATH_INFO: `/v3/users/8bb376a751024bdf8ef1d9e156cfdef3` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:12.161140 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f1c3b-2cfb-4a73-8d33-ef11b34a85c5 None admin] RBAC: Authorizing `identity:get_user(user_id=8bb376a751024bdf8ef1d9e156cfdef3)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:12.162631 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f1c3b-2cfb-4a73-8d33-ef11b34a85c5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:12.172263 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 529/1079] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:10:11 2022] GET /identity/v3/users/8bb376a751024bdf8ef1d9e156cfdef3 => generated 285 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:12.175778 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f1c3b-2cfb-4a73-8d33-ef11b34a85c5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:12.200477 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f95ce-297c-471e-a5bc-1a950625e8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:12.201751 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f95ce-297c-471e-a5bc-1a950625e8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:12.202270 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f95ce-297c-471e-a5bc-1a950625e8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:12.202761 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f95ce-297c-471e-a5bc-1a950625e8b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:12.210701 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:12.216794 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610f95ce-297c-471e-a5bc-1a950625e8b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:12.219713 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610f95ce-297c-471e-a5bc-1a950625e8b4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:12.239170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-610f95ce-297c-471e-a5bc-1a950625e8b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:10:12.239170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:10:12.239170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:10:12.239170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:10:12.239170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:10:12.239170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:10:12.239170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:10:12.239170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:10:12.239170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:10:12.239170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:10:12.239170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:10:12.239170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:10:12.239170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:10:12.239170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:10:12.239170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:10:12.240742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:10:12.240742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:10:12.240742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:10:12.240742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:10:12.240742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:10:12.240742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:10:12.240742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:10:12.240742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:10:12.240742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:10:12.240742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:10:12.240742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:10:12.240742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:10:12.240742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:10:12.240742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:10:12.240742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:10:12.240742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:10:12.240742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:10:12.242435 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:10:12.242435 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:10:12.242435 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:10:12.242435 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:10:12.242435 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:10:12.242435 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:10:12.242435 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:10:12.242435 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:10:12.242435 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 551/1080] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:10:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:10:12.248606 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f95ce-297c-471e-a5bc-1a950625e8b4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:12.271965 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:12.336929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f1c3b-2cfb-4a73-8d33-ef11b34a85c5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:12.383218 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1854e-c660-4b53-b918-ba2fe35c98bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:12.384246 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1854e-c660-4b53-b918-ba2fe35c98bd None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:12.384581 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1854e-c660-4b53-b918-ba2fe35c98bd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:12.384860 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1854e-c660-4b53-b918-ba2fe35c98bd None admin] PATH_INFO: `/v3/projects/2848e952fdde4cc885cce54b849bc549` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:12.386572 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada1854e-c660-4b53-b918-ba2fe35c98bd None admin] RBAC: Authorizing `identity:get_project(project_id=2848e952fdde4cc885cce54b849bc549)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:12.388109 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada1854e-c660-4b53-b918-ba2fe35c98bd None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:12.395090 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 530/1081] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:10:12 2022] GET /identity/v3/projects/2848e952fdde4cc885cce54b849bc549 => generated 311 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:10:12.396431 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1854e-c660-4b53-b918-ba2fe35c98bd None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:12.416209 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f95ce-297c-471e-a5bc-1a950625e8b4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:12.465002 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f2274-e40c-4386-a26f-802b8e26538b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:12.466507 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f2274-e40c-4386-a26f-802b8e26538b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:12.466981 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f2274-e40c-4386-a26f-802b8e26538b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:12.467498 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f2274-e40c-4386-a26f-802b8e26538b None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:12.469164 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0f2274-e40c-4386-a26f-802b8e26538b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:12.469749 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0f2274-e40c-4386-a26f-802b8e26538b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:12.472706 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0f2274-e40c-4386-a26f-802b8e26538b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:12.509896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 552/1082] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Jul 25 19:10:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:10:12.512933 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f2274-e40c-4386-a26f-802b8e26538b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:12.520140 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:12.539053 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1854e-c660-4b53-b918-ba2fe35c98bd None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:12.589073 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c478a-dcfb-4e32-8b0c-07f1eb0bb5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:12.590178 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c478a-dcfb-4e32-8b0c-07f1eb0bb5c5 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:12.590726 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c478a-dcfb-4e32-8b0c-07f1eb0bb5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:12.591478 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c478a-dcfb-4e32-8b0c-07f1eb0bb5c5 None admin] PATH_INFO: `/v3/projects/2848e952fdde4cc885cce54b849bc549/users/8bb376a751024bdf8ef1d9e156cfdef3/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:12.595053 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c478a-dcfb-4e32-8b0c-07f1eb0bb5c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=2848e952fdde4cc885cce54b849bc549, user_id=8bb376a751024bdf8ef1d9e156cfdef3, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:12.597918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c478a-dcfb-4e32-8b0c-07f1eb0bb5c5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:12.624651 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 531/1083] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Jul 25 19:10:12 2022] PUT /identity/v3/projects/2848e952fdde4cc885cce54b849bc549/users/8bb376a751024bdf8ef1d9e156cfdef3/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:10:12.640763 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:12.699856 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f2274-e40c-4386-a26f-802b8e26538b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:12.765119 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f7098-ad24-4628-b134-e5a546bd4799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:12.766570 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f7098-ad24-4628-b134-e5a546bd4799 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:12.767000 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f7098-ad24-4628-b134-e5a546bd4799 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:12.767425 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f7098-ad24-4628-b134-e5a546bd4799 None admin] PATH_INFO: `/v3/projects/191069caccd04233b012585e29cfa18b/users/860e7d26ad364795bb25836aadd4ac7b/roles/e1dc00443f96426e9edf6ea6ca8ae542` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:12.771323 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0f7098-ad24-4628-b134-e5a546bd4799 None admin] RBAC: Authorizing `identity:create_grant(project_id=191069caccd04233b012585e29cfa18b, user_id=860e7d26ad364795bb25836aadd4ac7b, role_id=e1dc00443f96426e9edf6ea6ca8ae542)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:12.773864 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0f7098-ad24-4628-b134-e5a546bd4799 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:12.795647 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 553/1084] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Jul 25 19:10:12 2022] PUT /identity/v3/projects/191069caccd04233b012585e29cfa18b/users/860e7d26ad364795bb25836aadd4ac7b/roles/e1dc00443f96426e9edf6ea6ca8ae542 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:10:14.937750 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6afe766-5fa3-437c-b3b5-2474bb51552e None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:14.938391 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6afe766-5fa3-437c-b3b5-2474bb51552e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:14.938741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6afe766-5fa3-437c-b3b5-2474bb51552e None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:14.941464 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 532/1085] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:10:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:10:14.950881 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed3ba5-5ac7-4527-aaef-687b62376805 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:14.951825 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed3ba5-5ac7-4527-aaef-687b62376805 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:14.951825 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed3ba5-5ac7-4527-aaef-687b62376805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:14.973696 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-48ed3ba5-5ac7-4527-aaef-687b62376805 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:10:14.975063 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-48ed3ba5-5ac7-4527-aaef-687b62376805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:15.070122 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 554/1086] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:10:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 19:10:15.098343 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6afe766-5fa3-437c-b3b5-2474bb51552e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:15.287333 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6afe766-5fa3-437c-b3b5-2474bb51552e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:15.342933 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f064f-8400-46b4-a60e-9e31aef74cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:15.343968 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f064f-8400-46b4-a60e-9e31aef74cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:15.344323 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f064f-8400-46b4-a60e-9e31aef74cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:15.344665 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f064f-8400-46b4-a60e-9e31aef74cf3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:15.360899 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f064f-8400-46b4-a60e-9e31aef74cf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:10:15.370325 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-5d3f064f-8400-46b4-a60e-9e31aef74cf3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:10:15.370325 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:10:15.370325 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:10:15.370325 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:10:15.370325 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:10:15.370325 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:10:15.370325 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:10:15.370325 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:10:15.370325 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:10:15.370325 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:10:15.370325 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:10:15.370325 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:10:15.370325 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:10:15.370325 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:10:15.370325 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:10:15.370325 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:10:15.372288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:10:15.372288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:10:15.372288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:10:15.372288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:10:15.372288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:10:15.372288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:10:15.372288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:10:15.372288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:10:15.372288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:10:15.372288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:10:15.372288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:10:15.372288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:10:15.372288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:10:15.372288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:10:15.372288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:10:15.372288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:10:15.374393 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:10:15.374393 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:10:15.374393 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:10:15.374393 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:10:15.374393 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:10:15.374393 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:10:15.374393 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:10:15.374393 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:10:15.374393 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:10:15.374393 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:10:15.374393 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:10:15.374393 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:10:15.376372 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 533/1087] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Jul 25 19:10:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:10:15.387407 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ed3ba5-5ac7-4527-aaef-687b62376805 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:15.516252 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:15.592088 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ed3ba5-5ac7-4527-aaef-687b62376805 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:15.662930 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138b2268-8c8f-4691-a3ec-e5a014d067d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:15.664291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b2268-8c8f-4691-a3ec-e5a014d067d9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:15.664727 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b2268-8c8f-4691-a3ec-e5a014d067d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:15.665286 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b2268-8c8f-4691-a3ec-e5a014d067d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:15.666580 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138b2268-8c8f-4691-a3ec-e5a014d067d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:15.667122 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138b2268-8c8f-4691-a3ec-e5a014d067d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:15.669742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138b2268-8c8f-4691-a3ec-e5a014d067d9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:15.687357 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 555/1088] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:10:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:15.697321 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f064f-8400-46b4-a60e-9e31aef74cf3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:15.718899 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:15.865004 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f064f-8400-46b4-a60e-9e31aef74cf3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:15.875467 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c09aa5-f429-4e3a-bc3d-a5ccf279978e None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:15.876015 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c09aa5-f429-4e3a-bc3d-a5ccf279978e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:15.876475 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c09aa5-f429-4e3a-bc3d-a5ccf279978e None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:15.878585 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 556/1089] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:10:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:10:15.887798 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734bc385-2cd5-4eaa-927d-b5b2c122d86e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:15.888229 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734bc385-2cd5-4eaa-927d-b5b2c122d86e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:15.888602 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734bc385-2cd5-4eaa-927d-b5b2c122d86e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:15.909782 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-734bc385-2cd5-4eaa-927d-b5b2c122d86e None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:10:15.911141 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-734bc385-2cd5-4eaa-927d-b5b2c122d86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:15.941639 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fdba2f-6bd2-4425-9370-b21f22cac8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:15.943243 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fdba2f-6bd2-4425-9370-b21f22cac8f3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:15.943830 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fdba2f-6bd2-4425-9370-b21f22cac8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:15.944310 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fdba2f-6bd2-4425-9370-b21f22cac8f3 None admin] PATH_INFO: `/v3/users/8bb376a751024bdf8ef1d9e156cfdef3` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:15.946686 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fdba2f-6bd2-4425-9370-b21f22cac8f3 None admin] RBAC: Authorizing `identity:get_user(user_id=8bb376a751024bdf8ef1d9e156cfdef3)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:15.949564 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fdba2f-6bd2-4425-9370-b21f22cac8f3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:15.954158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 534/1090] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:10:15 2022] GET /identity/v3/users/8bb376a751024bdf8ef1d9e156cfdef3 => generated 285 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:15.961594 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fdba2f-6bd2-4425-9370-b21f22cac8f3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:16.022560 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 557/1091] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:10:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:10:16.050595 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734bc385-2cd5-4eaa-927d-b5b2c122d86e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:16.117295 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:16.151083 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fdba2f-6bd2-4425-9370-b21f22cac8f3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:16.157230 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:16.209120 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b0bda3-d671-4f60-bea8-c47d8e501973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:16.209974 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0bda3-d671-4f60-bea8-c47d8e501973 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:16.210384 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0bda3-d671-4f60-bea8-c47d8e501973 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:16.210760 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0bda3-d671-4f60-bea8-c47d8e501973 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:16.253568 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734bc385-2cd5-4eaa-927d-b5b2c122d86e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:16.318681 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ab159-2f20-4ca5-9770-7ac8fac8ad6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:16.320170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ab159-2f20-4ca5-9770-7ac8fac8ad6a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:16.320682 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ab159-2f20-4ca5-9770-7ac8fac8ad6a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:16.321191 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ab159-2f20-4ca5-9770-7ac8fac8ad6a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:16.331968 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b0bda3-d671-4f60-bea8-c47d8e501973 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:16.333596 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b0bda3-d671-4f60-bea8-c47d8e501973 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:16.340679 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ab159-2f20-4ca5-9770-7ac8fac8ad6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:10:16.354173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-1f1ab159-2f20-4ca5-9770-7ac8fac8ad6a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 19:10:16.354173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:10:16.354173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:10:16.354173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:10:16.354173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:10:16.354173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:10:16.354173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:10:16.354173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:10:16.354173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:10:16.354173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:10:16.354173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:10:16.354173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:10:16.354173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:10:16.354173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:10:16.354173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:10:16.354173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:10:16.356440 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:10:16.356440 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:10:16.356440 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:10:16.356440 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:10:16.356440 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:10:16.356440 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:10:16.356440 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:10:16.356440 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:10:16.356440 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:10:16.356440 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:10:16.356440 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:10:16.356440 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:10:16.356440 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:10:16.356440 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:10:16.356440 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:10:16.356440 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:10:16.358240 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:10:16.358240 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:10:16.358240 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:10:16.358240 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:10:16.358240 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:10:16.358240 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:10:16.358240 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:10:16.358240 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:10:16.358240 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:10:16.358240 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:10:16.358240 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 19:10:16.358240 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:10:16.358240 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 558/1092] 10.222.0.18 () {56 vars in 1120 bytes} [Mon Jul 25 19:10:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:10:16.366959 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ab159-2f20-4ca5-9770-7ac8fac8ad6a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:16.491361 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:16.532133 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 535/1093] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:10:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 19:10:16.535188 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b0bda3-d671-4f60-bea8-c47d8e501973 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:16.539630 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ab159-2f20-4ca5-9770-7ac8fac8ad6a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:16.562121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:16.590420 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1dd0fb-87c8-4f0d-a4b6-683e05f562f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:16.591810 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1dd0fb-87c8-4f0d-a4b6-683e05f562f8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:16.592196 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1dd0fb-87c8-4f0d-a4b6-683e05f562f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:16.592487 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1dd0fb-87c8-4f0d-a4b6-683e05f562f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:16.593273 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1dd0fb-87c8-4f0d-a4b6-683e05f562f8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:16.593655 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1dd0fb-87c8-4f0d-a4b6-683e05f562f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:16.595395 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1dd0fb-87c8-4f0d-a4b6-683e05f562f8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:16.608944 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 559/1094] 10.222.0.18 () {56 vars in 1115 bytes} [Mon Jul 25 19:10:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:10:16.613695 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1dd0fb-87c8-4f0d-a4b6-683e05f562f8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:16.774204 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b0bda3-d671-4f60-bea8-c47d8e501973 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:16.777835 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1dd0fb-87c8-4f0d-a4b6-683e05f562f8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:16.828566 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f9426-4f15-4fca-b81d-437cb0ca2ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:16.829942 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f9426-4f15-4fca-b81d-437cb0ca2ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:16.830392 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f9426-4f15-4fca-b81d-437cb0ca2ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:16.830840 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f9426-4f15-4fca-b81d-437cb0ca2ec4 None admin] PATH_INFO: `/v3/projects/2848e952fdde4cc885cce54b849bc549` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:16.832932 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f9426-4f15-4fca-b81d-437cb0ca2ec4 None admin] RBAC: Authorizing `identity:get_project(project_id=2848e952fdde4cc885cce54b849bc549)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:16.835650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f9426-4f15-4fca-b81d-437cb0ca2ec4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:16.839385 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 536/1095] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:10:16 2022] GET /identity/v3/projects/2848e952fdde4cc885cce54b849bc549 => generated 311 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:10:16.846919 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e061c-1e63-4ac7-bb41-966c5d8386da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:16.848078 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e061c-1e63-4ac7-bb41-966c5d8386da None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:16.848414 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e061c-1e63-4ac7-bb41-966c5d8386da None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:16.848878 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e061c-1e63-4ac7-bb41-966c5d8386da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:16.850608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f9426-4f15-4fca-b81d-437cb0ca2ec4 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:16.865881 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581e061c-1e63-4ac7-bb41-966c5d8386da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:16.867648 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581e061c-1e63-4ac7-bb41-966c5d8386da None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:16.888597 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-581e061c-1e63-4ac7-bb41-966c5d8386da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:10:16.888597 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:10:16.888597 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:10:16.888597 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:10:16.888597 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:10:16.888597 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:10:16.888597 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:10:16.888597 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:10:16.888597 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:10:16.888597 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:10:16.888597 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:10:16.888597 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:10:16.888597 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:10:16.888597 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:10:16.888597 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:10:16.889699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:10:16.889699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:10:16.889699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:10:16.889699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:10:16.889699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:10:16.889699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:10:16.889699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:10:16.889699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:10:16.889699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:10:16.889699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:10:16.889699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:10:16.889699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:10:16.889699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:10:16.889699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:10:16.889699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:10:16.889699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:10:16.889699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:10:16.890623 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:10:16.890623 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:10:16.890623 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:10:16.890623 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:10:16.890623 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 560/1096] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:10:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:10:16.895882 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e061c-1e63-4ac7-bb41-966c5d8386da None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:16.936565 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:16.941581 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:17.046301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f9426-4f15-4fca-b81d-437cb0ca2ec4 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:17.064446 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e061c-1e63-4ac7-bb41-966c5d8386da None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:17.128119 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeffb24-57d4-451c-b7bf-3cc398046eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:17.129043 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeffb24-57d4-451c-b7bf-3cc398046eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:17.129373 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeffb24-57d4-451c-b7bf-3cc398046eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:17.129820 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeffb24-57d4-451c-b7bf-3cc398046eb1 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:17.131121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeffb24-57d4-451c-b7bf-3cc398046eb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:17.131650 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeffb24-57d4-451c-b7bf-3cc398046eb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:17.132099 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a17c9fc-b9b7-4f09-ae68-bcefa0dd2782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:17.133184 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a17c9fc-b9b7-4f09-ae68-bcefa0dd2782 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:17.133595 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a17c9fc-b9b7-4f09-ae68-bcefa0dd2782 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:17.134819 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a17c9fc-b9b7-4f09-ae68-bcefa0dd2782 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:17.135477 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeffb24-57d4-451c-b7bf-3cc398046eb1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:17.136479 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a17c9fc-b9b7-4f09-ae68-bcefa0dd2782 None admin] RBAC: Adding query filter params (role.id=ed7fec18968241efa76a0ee769d96da9, scope.project.id=2848e952fdde4cc885cce54b849bc549, user.id=8bb376a751024bdf8ef1d9e156cfdef3) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:17.137048 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a17c9fc-b9b7-4f09-ae68-bcefa0dd2782 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:17.139865 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a17c9fc-b9b7-4f09-ae68-bcefa0dd2782 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:17.152595 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 537/1097] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:10:16 2022] GET /identity/v3/role_assignments?user.id=8bb376a751024bdf8ef1d9e156cfdef3&scope.project.id=2848e952fdde4cc885cce54b849bc549&role.id=ed7fec18968241efa76a0ee769d96da9 => generated 608 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:17.157416 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 561/1098] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:10:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:10:17.164684 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a17c9fc-b9b7-4f09-ae68-bcefa0dd2782 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:17.349360 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a17c9fc-b9b7-4f09-ae68-bcefa0dd2782 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:17.369738 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:17.412479 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f5a035-d0f6-422d-aec3-749e41eeb368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:17.413385 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f5a035-d0f6-422d-aec3-749e41eeb368 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:17.413709 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f5a035-d0f6-422d-aec3-749e41eeb368 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:17.413994 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f5a035-d0f6-422d-aec3-749e41eeb368 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:17.425693 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f5a035-d0f6-422d-aec3-749e41eeb368 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:17.427337 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f5a035-d0f6-422d-aec3-749e41eeb368 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:17.459346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-14f5a035-d0f6-422d-aec3-749e41eeb368 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 19:10:17.459346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:10:17.459346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:10:17.459346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:10:17.459346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:10:17.459346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:10:17.459346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:10:17.459346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:10:17.459346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:10:17.459346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:10:17.459346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:10:17.459346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:10:17.459346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:10:17.459346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:10:17.459346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:10:17.459346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:10:17.461374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:10:17.461374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:10:17.461374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:10:17.461374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:10:17.461374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:10:17.461374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:10:17.461374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:10:17.461374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:10:17.461374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:10:17.461374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:10:17.461374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:10:17.461374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:10:17.461374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:10:17.461374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:10:17.461374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:10:17.461374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:10:17.461374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:10:17.461374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:10:17.463251 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:10:17.463251 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:10:17.463251 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:10:17.463251 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:10:17.463251 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:10:17.463251 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:10:17.463251 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:10:17.463251 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:10:17.463251 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 19:10:17.463251 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:10:17.463251 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 538/1099] 10.222.0.18 () {56 vars in 1093 bytes} [Mon Jul 25 19:10:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:10:17.472424 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeffb24-57d4-451c-b7bf-3cc398046eb1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:17.664396 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:17.691240 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeffb24-57d4-451c-b7bf-3cc398046eb1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:17.754688 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b7993-ace1-4bff-a5ef-28f082458bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:17.755898 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b7993-ace1-4bff-a5ef-28f082458bfd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:17.756401 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b7993-ace1-4bff-a5ef-28f082458bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:17.756889 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b7993-ace1-4bff-a5ef-28f082458bfd None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:17.758533 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b7993-ace1-4bff-a5ef-28f082458bfd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:17.759327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b7993-ace1-4bff-a5ef-28f082458bfd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:17.762850 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b7993-ace1-4bff-a5ef-28f082458bfd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:17.798656 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 562/1100] 10.222.0.18 () {56 vars in 1133 bytes} [Mon Jul 25 19:10:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:17.806967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f5a035-d0f6-422d-aec3-749e41eeb368 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:18.053409 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f5a035-d0f6-422d-aec3-749e41eeb368 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:18.125423 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35873a-397c-43f5-92cb-7f0f257b2d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:18.126540 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35873a-397c-43f5-92cb-7f0f257b2d5d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:18.127006 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35873a-397c-43f5-92cb-7f0f257b2d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:18.127478 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35873a-397c-43f5-92cb-7f0f257b2d5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:18.191198 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:18.268909 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c35873a-397c-43f5-92cb-7f0f257b2d5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:18.271791 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c35873a-397c-43f5-92cb-7f0f257b2d5d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:18.485909 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 539/1101] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:10:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 675 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 19:10:18.487832 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b7993-ace1-4bff-a5ef-28f082458bfd None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:18.686689 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b7993-ace1-4bff-a5ef-28f082458bfd None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:18.756973 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ac454-22d2-4adf-a67e-2786a251af53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:18.758271 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ac454-22d2-4adf-a67e-2786a251af53 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:18.759377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ac454-22d2-4adf-a67e-2786a251af53 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:18.759970 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ac454-22d2-4adf-a67e-2786a251af53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:18.809362 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ac454-22d2-4adf-a67e-2786a251af53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:18.812389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ac454-22d2-4adf-a67e-2786a251af53 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:18.830103 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:18.852609 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-184ac454-22d2-4adf-a67e-2786a251af53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:10:18.852609 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:10:18.852609 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:10:18.852609 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:10:18.852609 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:10:18.852609 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:10:18.852609 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:10:18.852609 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:10:18.852609 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:10:18.852609 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:10:18.852609 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:10:18.852609 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:10:18.852609 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:10:18.852609 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:10:18.852609 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:10:18.854286 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:10:18.854286 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:10:18.854286 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:10:18.854286 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:10:18.854286 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:10:18.854286 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:10:18.854286 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:10:18.854286 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:10:18.854286 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:10:18.854286 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:10:18.854286 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:10:18.854286 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:10:18.854286 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:10:18.854286 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:10:18.854286 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:10:18.854286 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:10:18.854286 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:10:18.856491 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:10:18.856491 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:10:18.856491 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:10:18.856491 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:10:18.856491 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 563/1102] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:10:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:10:18.865120 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35873a-397c-43f5-92cb-7f0f257b2d5d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:19.003930 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:19.071887 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35873a-397c-43f5-92cb-7f0f257b2d5d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:19.147582 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bfd176-5ac6-4d81-abbb-9fd305e76775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:19.148761 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfd176-5ac6-4d81-abbb-9fd305e76775 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:19.149273 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfd176-5ac6-4d81-abbb-9fd305e76775 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:19.150314 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfd176-5ac6-4d81-abbb-9fd305e76775 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:19.151779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bfd176-5ac6-4d81-abbb-9fd305e76775 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:19.152493 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bfd176-5ac6-4d81-abbb-9fd305e76775 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:19.155151 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bfd176-5ac6-4d81-abbb-9fd305e76775 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:19.180112 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 540/1103] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:10:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:19.191044 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ac454-22d2-4adf-a67e-2786a251af53 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:19.375437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ac454-22d2-4adf-a67e-2786a251af53 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:19.450306 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b83ab-1b05-46bf-9a6a-250296adba00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:19.451780 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b83ab-1b05-46bf-9a6a-250296adba00 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:19.452267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b83ab-1b05-46bf-9a6a-250296adba00 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:19.452752 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b83ab-1b05-46bf-9a6a-250296adba00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:19.468527 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322b83ab-1b05-46bf-9a6a-250296adba00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:19.471501 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322b83ab-1b05-46bf-9a6a-250296adba00 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:19.498917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-322b83ab-1b05-46bf-9a6a-250296adba00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:10:19.498917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:10:19.498917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:10:19.498917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:10:19.498917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:10:19.498917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:10:19.498917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:10:19.498917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:10:19.498917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:10:19.498917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:10:19.498917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:10:19.498917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:10:19.498917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:10:19.498917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:10:19.498917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:10:19.501812 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:10:19.501812 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:10:19.501812 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:10:19.501812 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:10:19.501812 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:10:19.501812 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:10:19.501812 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:10:19.501812 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:10:19.501812 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:10:19.501812 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:10:19.501812 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:10:19.501812 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:10:19.501812 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:10:19.501812 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:10:19.501812 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:10:19.501812 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:10:19.501812 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:10:19.504079 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:10:19.504079 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:10:19.504079 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:10:19.504079 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:10:19.504079 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:10:19.504079 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:10:19.504079 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:10:19.504079 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:10:19.504079 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 564/1104] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:10:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:10:19.512057 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bfd176-5ac6-4d81-abbb-9fd305e76775 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:19.679413 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bfd176-5ac6-4d81-abbb-9fd305e76775 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:19.747616 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:19.768751 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746f2aa-b6e9-4aa9-a93d-95538fc6a6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:19.769957 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746f2aa-b6e9-4aa9-a93d-95538fc6a6b8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:19.770459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746f2aa-b6e9-4aa9-a93d-95538fc6a6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:19.770956 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746f2aa-b6e9-4aa9-a93d-95538fc6a6b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:19.772563 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746f2aa-b6e9-4aa9-a93d-95538fc6a6b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:19.773130 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746f2aa-b6e9-4aa9-a93d-95538fc6a6b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:19.776230 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746f2aa-b6e9-4aa9-a93d-95538fc6a6b8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:19.809972 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 541/1105] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Jul 25 19:10:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:19.811369 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b83ab-1b05-46bf-9a6a-250296adba00 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:19.907895 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:19.947620 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed184e44-d43a-4c30-8806-4eb744e0323a None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:19.948004 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed184e44-d43a-4c30-8806-4eb744e0323a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:19.948499 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed184e44-d43a-4c30-8806-4eb744e0323a None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:19.951154 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 542/1106] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:10:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:10:19.958706 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472a48d-c8db-419e-ac8a-9be0f83ba9e1 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:19.959035 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472a48d-c8db-419e-ac8a-9be0f83ba9e1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:19.959364 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472a48d-c8db-419e-ac8a-9be0f83ba9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:19.978166 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b83ab-1b05-46bf-9a6a-250296adba00 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:19.983804 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8472a48d-c8db-419e-ac8a-9be0f83ba9e1 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:10:19.986683 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8472a48d-c8db-419e-ac8a-9be0f83ba9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:20.043263 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53910f-033a-4ffa-8f56-d4f463c12352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:20.044475 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53910f-033a-4ffa-8f56-d4f463c12352 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:20.044950 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53910f-033a-4ffa-8f56-d4f463c12352 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:20.045454 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53910f-033a-4ffa-8f56-d4f463c12352 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:20.047112 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f53910f-033a-4ffa-8f56-d4f463c12352 None admin] RBAC: Adding query filter params (role.id=e1dc00443f96426e9edf6ea6ca8ae542, scope.project.id=191069caccd04233b012585e29cfa18b, user.id=860e7d26ad364795bb25836aadd4ac7b) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:20.047705 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f53910f-033a-4ffa-8f56-d4f463c12352 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:20.050477 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f53910f-033a-4ffa-8f56-d4f463c12352 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:20.075438 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 565/1107] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:10:19 2022] GET /identity/v3/role_assignments?user.id=860e7d26ad364795bb25836aadd4ac7b&scope.project.id=191069caccd04233b012585e29cfa18b&role.id=e1dc00443f96426e9edf6ea6ca8ae542 => generated 608 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:20.101406 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 543/1108] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:10:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 19:10:20.146450 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53910f-033a-4ffa-8f56-d4f463c12352 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:20.358600 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53910f-033a-4ffa-8f56-d4f463c12352 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:20.402853 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:20.427348 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9811a0b0-4780-4175-840a-abd5a263987d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:20.428759 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9811a0b0-4780-4175-840a-abd5a263987d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:20.429269 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9811a0b0-4780-4175-840a-abd5a263987d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:20.429729 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9811a0b0-4780-4175-840a-abd5a263987d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:20.433535 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9811a0b0-4780-4175-840a-abd5a263987d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:20.436349 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9811a0b0-4780-4175-840a-abd5a263987d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:20.438409 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 566/1109] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:10:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:20.449128 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472a48d-c8db-419e-ac8a-9be0f83ba9e1 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:20.655708 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472a48d-c8db-419e-ac8a-9be0f83ba9e1 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:20.705608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:20.735879 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e96057-e0ba-4daa-9986-df86d829982d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:20.737057 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e96057-e0ba-4daa-9986-df86d829982d None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:20.737598 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e96057-e0ba-4daa-9986-df86d829982d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:20.738104 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e96057-e0ba-4daa-9986-df86d829982d None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:20.739563 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e96057-e0ba-4daa-9986-df86d829982d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:20.742441 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e96057-e0ba-4daa-9986-df86d829982d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:20.787588 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 544/1110] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Jul 25 19:10:20 2022] POST /identity/v3/users => generated 286 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:10:23.357589 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdd19a-b5aa-49a3-a846-3bc13cdb26d6 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:23.358345 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdd19a-b5aa-49a3-a846-3bc13cdb26d6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:23.358345 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdd19a-b5aa-49a3-a846-3bc13cdb26d6 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:23.359601 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 567/1111] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:10:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:10:23.366380 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60420364-9bae-4363-a0c2-f89cdd2650d0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:23.367105 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60420364-9bae-4363-a0c2-f89cdd2650d0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:23.367105 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60420364-9bae-4363-a0c2-f89cdd2650d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:23.391359 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-60420364-9bae-4363-a0c2-f89cdd2650d0 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:10:23.392878 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-60420364-9bae-4363-a0c2-f89cdd2650d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:23.484832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 545/1112] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:10:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 19:10:23.510766 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cdd19a-b5aa-49a3-a846-3bc13cdb26d6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:23.667141 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cdd19a-b5aa-49a3-a846-3bc13cdb26d6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:23.709883 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881ec09-9080-4a48-9bdf-3be6b46232ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:23.710877 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881ec09-9080-4a48-9bdf-3be6b46232ab None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:23.711205 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881ec09-9080-4a48-9bdf-3be6b46232ab None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:23.711665 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881ec09-9080-4a48-9bdf-3be6b46232ab None admin] PATH_INFO: `/v3/roles/191e7e31dad148b7a0eb52d3afb5e50e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:23.713997 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f881ec09-9080-4a48-9bdf-3be6b46232ab None admin] RBAC: Authorizing `identity:get_role(role_id=191e7e31dad148b7a0eb52d3afb5e50e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:23.715808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f881ec09-9080-4a48-9bdf-3be6b46232ab None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:23.717710 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 568/1113] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:10:23 2022] GET /identity/v3/roles/191e7e31dad148b7a0eb52d3afb5e50e => generated 223 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:23.724100 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60420364-9bae-4363-a0c2-f89cdd2650d0 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:23.886246 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60420364-9bae-4363-a0c2-f89cdd2650d0 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:23.893684 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:23.938903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7f502-7555-4296-9bb1-a6c4e36e8cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:23.940513 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7f502-7555-4296-9bb1-a6c4e36e8cbb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:23.941002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7f502-7555-4296-9bb1-a6c4e36e8cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:23.941455 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7f502-7555-4296-9bb1-a6c4e36e8cbb None admin] PATH_INFO: `/v3/users/a84cef0f156a4c15a027ceb7cf374199` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:23.967116 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c7f502-7555-4296-9bb1-a6c4e36e8cbb None admin] RBAC: Authorizing `identity:get_user(user_id=a84cef0f156a4c15a027ceb7cf374199)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:23.969886 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c7f502-7555-4296-9bb1-a6c4e36e8cbb None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:23.972890 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 546/1114] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:10:23 2022] GET /identity/v3/users/a84cef0f156a4c15a027ceb7cf374199 => generated 286 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:23.981156 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881ec09-9080-4a48-9bdf-3be6b46232ab None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:24.076583 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:24.158892 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881ec09-9080-4a48-9bdf-3be6b46232ab None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:24.212522 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e1cde-b00f-4054-ad9f-54fa0662b7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:24.213275 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e1cde-b00f-4054-ad9f-54fa0662b7ba None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:24.213604 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e1cde-b00f-4054-ad9f-54fa0662b7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:24.213926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e1cde-b00f-4054-ad9f-54fa0662b7ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:24.340910 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e1cde-b00f-4054-ad9f-54fa0662b7ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:24.342876 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e1cde-b00f-4054-ad9f-54fa0662b7ba None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:24.449135 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:24.521632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 569/1115] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:10:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 19:10:24.528910 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7f502-7555-4296-9bb1-a6c4e36e8cbb None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:24.708074 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7f502-7555-4296-9bb1-a6c4e36e8cbb None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:24.759385 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5c3cc-afb6-4ffd-941e-fa64b94b9dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:24.761069 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5c3cc-afb6-4ffd-941e-fa64b94b9dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:24.761576 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5c3cc-afb6-4ffd-941e-fa64b94b9dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:24.762070 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5c3cc-afb6-4ffd-941e-fa64b94b9dd3 None admin] PATH_INFO: `/v3/projects/2848e952fdde4cc885cce54b849bc549` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:24.764268 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba5c3cc-afb6-4ffd-941e-fa64b94b9dd3 None admin] RBAC: Authorizing `identity:get_project(project_id=2848e952fdde4cc885cce54b849bc549)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:24.767042 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba5c3cc-afb6-4ffd-941e-fa64b94b9dd3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:24.770386 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 547/1116] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:10:24 2022] GET /identity/v3/projects/2848e952fdde4cc885cce54b849bc549 => generated 311 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:24.780793 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e1cde-b00f-4054-ad9f-54fa0662b7ba None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:24.935753 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e1cde-b00f-4054-ad9f-54fa0662b7ba None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:24.986645 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a3b9d8-767f-4572-a697-9ef7f3cb1527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:24.987304 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3b9d8-767f-4572-a697-9ef7f3cb1527 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:24.987666 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3b9d8-767f-4572-a697-9ef7f3cb1527 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:24.987999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3b9d8-767f-4572-a697-9ef7f3cb1527 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:24.988911 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a3b9d8-767f-4572-a697-9ef7f3cb1527 None admin] RBAC: Adding query filter params (role.id=191e7e31dad148b7a0eb52d3afb5e50e, scope.project.id=2848e952fdde4cc885cce54b849bc549, user.id=a84cef0f156a4c15a027ceb7cf374199) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:24.989273 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a3b9d8-767f-4572-a697-9ef7f3cb1527 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:24.990894 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a3b9d8-767f-4572-a697-9ef7f3cb1527 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:25.001020 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 570/1117] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:10:24 2022] GET /identity/v3/role_assignments?user.id=a84cef0f156a4c15a027ceb7cf374199&scope.project.id=2848e952fdde4cc885cce54b849bc549&role.id=191e7e31dad148b7a0eb52d3afb5e50e => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:27.320369 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528b11a-0970-47f2-ad5c-c9d430864a6d None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:27.321191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528b11a-0970-47f2-ad5c-c9d430864a6d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:27.321191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528b11a-0970-47f2-ad5c-c9d430864a6d None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:27.322741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 548/1118] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:10:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:10:27.329116 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cc379-e05a-48c6-be42-73570464efac None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:27.329713 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cc379-e05a-48c6-be42-73570464efac None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:27.329713 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cc379-e05a-48c6-be42-73570464efac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:27.351136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-097cc379-e05a-48c6-be42-73570464efac None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:10:27.352555 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-097cc379-e05a-48c6-be42-73570464efac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:27.377521 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:27.446919 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 571/1119] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:10:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 19:10:27.471026 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528b11a-0970-47f2-ad5c-c9d430864a6d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:27.510321 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:27.609960 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528b11a-0970-47f2-ad5c-c9d430864a6d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:27.658922 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4093ad0-3672-4094-8aa2-627af45caf2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:27.659850 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4093ad0-3672-4094-8aa2-627af45caf2f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:27.660171 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4093ad0-3672-4094-8aa2-627af45caf2f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:27.660445 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4093ad0-3672-4094-8aa2-627af45caf2f None admin] PATH_INFO: `/v3/roles/191e7e31dad148b7a0eb52d3afb5e50e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:27.662248 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4093ad0-3672-4094-8aa2-627af45caf2f None admin] RBAC: Authorizing `identity:get_role(role_id=191e7e31dad148b7a0eb52d3afb5e50e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:27.663692 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4093ad0-3672-4094-8aa2-627af45caf2f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:27.665215 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 549/1120] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:10:27 2022] GET /identity/v3/roles/191e7e31dad148b7a0eb52d3afb5e50e => generated 223 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:10:27.672665 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097cc379-e05a-48c6-be42-73570464efac None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:27.840305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097cc379-e05a-48c6-be42-73570464efac None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:27.908497 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8897a21-8d8f-4934-97e4-ee1dfa2096cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:27.910038 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8897a21-8d8f-4934-97e4-ee1dfa2096cf None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:27.910519 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8897a21-8d8f-4934-97e4-ee1dfa2096cf None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:27.911000 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8897a21-8d8f-4934-97e4-ee1dfa2096cf None admin] PATH_INFO: `/v3/users/a84cef0f156a4c15a027ceb7cf374199` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:27.913312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8897a21-8d8f-4934-97e4-ee1dfa2096cf None admin] RBAC: Authorizing `identity:get_user(user_id=a84cef0f156a4c15a027ceb7cf374199)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:27.916462 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8897a21-8d8f-4934-97e4-ee1dfa2096cf None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:27.919702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 572/1121] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:10:27 2022] GET /identity/v3/users/a84cef0f156a4c15a027ceb7cf374199 => generated 286 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:27.929128 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4093ad0-3672-4094-8aa2-627af45caf2f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:28.090708 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:28.096459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4093ad0-3672-4094-8aa2-627af45caf2f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:28.138514 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe580f-7f14-43bd-839e-c78667c05724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:28.139407 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe580f-7f14-43bd-839e-c78667c05724 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:28.139722 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe580f-7f14-43bd-839e-c78667c05724 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:28.140001 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe580f-7f14-43bd-839e-c78667c05724 None admin] PATH_INFO: `/v3/projects/2848e952fdde4cc885cce54b849bc549` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:28.141321 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fe580f-7f14-43bd-839e-c78667c05724 None admin] RBAC: Authorizing `identity:get_project(project_id=2848e952fdde4cc885cce54b849bc549)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:28.142818 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fe580f-7f14-43bd-839e-c78667c05724 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:28.144705 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 550/1122] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:10:27 2022] GET /identity/v3/projects/2848e952fdde4cc885cce54b849bc549 => generated 311 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:28.153470 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8897a21-8d8f-4934-97e4-ee1dfa2096cf None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:28.186589 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:28.326559 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8897a21-8d8f-4934-97e4-ee1dfa2096cf None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:28.396202 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a199137e-6974-4dd7-8bdd-fb4909d5425d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:28.397660 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199137e-6974-4dd7-8bdd-fb4909d5425d None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:28.398150 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199137e-6974-4dd7-8bdd-fb4909d5425d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:28.398630 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199137e-6974-4dd7-8bdd-fb4909d5425d None admin] PATH_INFO: `/v3/projects/2848e952fdde4cc885cce54b849bc549/users/a84cef0f156a4c15a027ceb7cf374199/roles/191e7e31dad148b7a0eb52d3afb5e50e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:28.404055 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a199137e-6974-4dd7-8bdd-fb4909d5425d None admin] RBAC: Authorizing `identity:create_grant(project_id=2848e952fdde4cc885cce54b849bc549, user_id=a84cef0f156a4c15a027ceb7cf374199, role_id=191e7e31dad148b7a0eb52d3afb5e50e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:28.406730 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a199137e-6974-4dd7-8bdd-fb4909d5425d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:28.429747 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 573/1123] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Jul 25 19:10:28 2022] PUT /identity/v3/projects/2848e952fdde4cc885cce54b849bc549/users/a84cef0f156a4c15a027ceb7cf374199/roles/191e7e31dad148b7a0eb52d3afb5e50e => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:10:31.240896 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7a575-7511-4828-b780-4ce4bd83aeaa None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:31.241697 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7a575-7511-4828-b780-4ce4bd83aeaa None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:31.241697 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7a575-7511-4828-b780-4ce4bd83aeaa None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:31.243732 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 551/1124] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:10:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:10:31.252984 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944efb8a-02fe-4981-b246-fb966fe4d427 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:31.253717 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944efb8a-02fe-4981-b246-fb966fe4d427 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:31.253717 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944efb8a-02fe-4981-b246-fb966fe4d427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:31.278835 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-944efb8a-02fe-4981-b246-fb966fe4d427 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:10:31.280917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-944efb8a-02fe-4981-b246-fb966fe4d427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:31.394630 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 574/1125] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:10:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 19:10:31.439604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e7a575-7511-4828-b780-4ce4bd83aeaa None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:31.599465 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e7a575-7511-4828-b780-4ce4bd83aeaa None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:31.658969 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce0362-1944-4e88-bf84-d293d1717f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:31.661857 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce0362-1944-4e88-bf84-d293d1717f09 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:31.662926 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce0362-1944-4e88-bf84-d293d1717f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:31.663400 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce0362-1944-4e88-bf84-d293d1717f09 None admin] PATH_INFO: `/v3/roles/191e7e31dad148b7a0eb52d3afb5e50e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:31.667495 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce0362-1944-4e88-bf84-d293d1717f09 None admin] RBAC: Authorizing `identity:get_role(role_id=191e7e31dad148b7a0eb52d3afb5e50e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:31.672551 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce0362-1944-4e88-bf84-d293d1717f09 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:31.675486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 552/1126] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:10:31 2022] GET /identity/v3/roles/191e7e31dad148b7a0eb52d3afb5e50e => generated 223 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:31.687018 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944efb8a-02fe-4981-b246-fb966fe4d427 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:31.710983 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:31.913330 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944efb8a-02fe-4981-b246-fb966fe4d427 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:31.968753 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4120fd-7b63-46cf-89ff-0952f4b32a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:31.969665 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4120fd-7b63-46cf-89ff-0952f4b32a62 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:31.970006 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4120fd-7b63-46cf-89ff-0952f4b32a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:31.970344 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4120fd-7b63-46cf-89ff-0952f4b32a62 None admin] PATH_INFO: `/v3/users/a84cef0f156a4c15a027ceb7cf374199` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:31.975556 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4120fd-7b63-46cf-89ff-0952f4b32a62 None admin] RBAC: Authorizing `identity:get_user(user_id=a84cef0f156a4c15a027ceb7cf374199)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:31.975556 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4120fd-7b63-46cf-89ff-0952f4b32a62 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:31.975994 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 575/1127] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:10:31 2022] GET /identity/v3/users/a84cef0f156a4c15a027ceb7cf374199 => generated 286 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:31.983299 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce0362-1944-4e88-bf84-d293d1717f09 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:32.058248 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:32.181816 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce0362-1944-4e88-bf84-d293d1717f09 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:32.273606 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e69e7c-de30-48db-b789-17083dfdc8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:32.274570 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e69e7c-de30-48db-b789-17083dfdc8a2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:32.274570 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e69e7c-de30-48db-b789-17083dfdc8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:32.274757 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e69e7c-de30-48db-b789-17083dfdc8a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:32.375831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e69e7c-de30-48db-b789-17083dfdc8a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:32.377478 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e69e7c-de30-48db-b789-17083dfdc8a2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:32.423085 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:32.509340 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 553/1128] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:10:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 19:10:32.517044 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4120fd-7b63-46cf-89ff-0952f4b32a62 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:32.650164 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:32.677973 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4120fd-7b63-46cf-89ff-0952f4b32a62 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:32.732475 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b40b2-6a29-43d1-a9b7-d766a16fc1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:32.733972 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b40b2-6a29-43d1-a9b7-d766a16fc1fd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:32.734461 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b40b2-6a29-43d1-a9b7-d766a16fc1fd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:32.734959 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b40b2-6a29-43d1-a9b7-d766a16fc1fd None admin] PATH_INFO: `/v3/projects/2848e952fdde4cc885cce54b849bc549` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:32.737370 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b40b2-6a29-43d1-a9b7-d766a16fc1fd None admin] RBAC: Authorizing `identity:get_project(project_id=2848e952fdde4cc885cce54b849bc549)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:32.740216 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b40b2-6a29-43d1-a9b7-d766a16fc1fd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:32.744407 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 576/1129] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:10:32 2022] GET /identity/v3/projects/2848e952fdde4cc885cce54b849bc549 => generated 311 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:32.751166 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e69e7c-de30-48db-b789-17083dfdc8a2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:32.904996 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e69e7c-de30-48db-b789-17083dfdc8a2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:32.948402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845aed1-bd39-4d4c-b475-7d98cd56c758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:32.949148 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845aed1-bd39-4d4c-b475-7d98cd56c758 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:32.949605 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845aed1-bd39-4d4c-b475-7d98cd56c758 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:32.949768 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845aed1-bd39-4d4c-b475-7d98cd56c758 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:32.950700 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9845aed1-bd39-4d4c-b475-7d98cd56c758 None admin] RBAC: Adding query filter params (role.id=191e7e31dad148b7a0eb52d3afb5e50e, scope.project.id=2848e952fdde4cc885cce54b849bc549, user.id=a84cef0f156a4c15a027ceb7cf374199) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:32.951114 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9845aed1-bd39-4d4c-b475-7d98cd56c758 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:32.952733 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9845aed1-bd39-4d4c-b475-7d98cd56c758 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:32.955268 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:32.961279 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 554/1130] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:10:32 2022] GET /identity/v3/role_assignments?user.id=a84cef0f156a4c15a027ceb7cf374199&scope.project.id=2848e952fdde4cc885cce54b849bc549&role.id=191e7e31dad148b7a0eb52d3afb5e50e => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:36.259811 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef83fdf-dfad-48a2-a68c-f3fdc878394e None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:36.260421 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef83fdf-dfad-48a2-a68c-f3fdc878394e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:36.260421 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef83fdf-dfad-48a2-a68c-f3fdc878394e None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:36.261845 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 577/1131] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:10:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:10:36.269374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001bb2e-49eb-4e89-abaa-864417944153 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:36.270246 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001bb2e-49eb-4e89-abaa-864417944153 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:36.270246 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001bb2e-49eb-4e89-abaa-864417944153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:36.291554 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b001bb2e-49eb-4e89-abaa-864417944153 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:10:36.292946 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b001bb2e-49eb-4e89-abaa-864417944153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:36.382595 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 555/1132] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:10:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:10:36.412006 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef83fdf-dfad-48a2-a68c-f3fdc878394e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:36.570893 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef83fdf-dfad-48a2-a68c-f3fdc878394e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:36.638066 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75b422-7ff5-4101-bdb6-93e2618474a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:36.639322 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75b422-7ff5-4101-bdb6-93e2618474a5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:36.639768 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75b422-7ff5-4101-bdb6-93e2618474a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:36.640176 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75b422-7ff5-4101-bdb6-93e2618474a5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:36.642234 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75b422-7ff5-4101-bdb6-93e2618474a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:36.645141 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75b422-7ff5-4101-bdb6-93e2618474a5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:36.648500 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 578/1133] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:10:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:36.656030 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b001bb2e-49eb-4e89-abaa-864417944153 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:36.817591 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b001bb2e-49eb-4e89-abaa-864417944153 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:36.871214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:36.884777 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a662a-d398-469d-88a7-556c4f6419af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:36.885546 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a662a-d398-469d-88a7-556c4f6419af None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:36.885887 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a662a-d398-469d-88a7-556c4f6419af None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:36.886208 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a662a-d398-469d-88a7-556c4f6419af None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:36.887048 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57a662a-d398-469d-88a7-556c4f6419af None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:36.888725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57a662a-d398-469d-88a7-556c4f6419af None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:36.927176 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 556/1134] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Jul 25 19:10:36 2022] POST /identity/v3/projects => generated 311 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:10:39.355715 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7704988-b1ae-4edf-8ef5-0004e4409de9 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:39.356528 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7704988-b1ae-4edf-8ef5-0004e4409de9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:39.356692 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7704988-b1ae-4edf-8ef5-0004e4409de9 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:39.358255 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 579/1135] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:10:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:10:39.365019 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ad655-fbea-4d38-95c7-25ee14eafeba None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:39.365880 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ad655-fbea-4d38-95c7-25ee14eafeba None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:39.366050 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ad655-fbea-4d38-95c7-25ee14eafeba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:39.392358 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-185ad655-fbea-4d38-95c7-25ee14eafeba None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:10:39.394012 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-185ad655-fbea-4d38-95c7-25ee14eafeba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:39.495940 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 557/1136] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:10:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 19:10:39.521667 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7704988-b1ae-4edf-8ef5-0004e4409de9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:39.541506 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:39.660572 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7704988-b1ae-4edf-8ef5-0004e4409de9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:39.706593 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f1d659-f6d1-449c-bd61-7fa004c6a9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:39.707609 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f1d659-f6d1-449c-bd61-7fa004c6a9e0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:39.708778 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f1d659-f6d1-449c-bd61-7fa004c6a9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:39.708778 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f1d659-f6d1-449c-bd61-7fa004c6a9e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:39.710144 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f1d659-f6d1-449c-bd61-7fa004c6a9e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:39.712150 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f1d659-f6d1-449c-bd61-7fa004c6a9e0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:39.714621 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 580/1137] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:10:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:39.722607 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ad655-fbea-4d38-95c7-25ee14eafeba None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:39.881913 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ad655-fbea-4d38-95c7-25ee14eafeba None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:39.905636 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:39.953932 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfbc34e-e46a-4149-bb54-344f14a2a47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:39.955140 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfbc34e-e46a-4149-bb54-344f14a2a47b None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:39.955736 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfbc34e-e46a-4149-bb54-344f14a2a47b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:39.956165 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfbc34e-e46a-4149-bb54-344f14a2a47b None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:39.957570 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfbc34e-e46a-4149-bb54-344f14a2a47b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:39.960469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfbc34e-e46a-4149-bb54-344f14a2a47b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:40.001306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 558/1138] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Jul 25 19:10:39 2022] POST /identity/v3/users => generated 286 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:10:42.797466 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ffd5a-7bf8-4dc6-9b71-5c86bafcc173 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:42.798114 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ffd5a-7bf8-4dc6-9b71-5c86bafcc173 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:42.798114 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ffd5a-7bf8-4dc6-9b71-5c86bafcc173 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:42.799776 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 581/1139] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:10:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:10:42.807620 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8023b-2c83-4caa-949b-c12520d80d8c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:42.809996 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8023b-2c83-4caa-949b-c12520d80d8c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:42.810491 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8023b-2c83-4caa-949b-c12520d80d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:42.843445 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1df8023b-2c83-4caa-949b-c12520d80d8c None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:10:42.844848 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1df8023b-2c83-4caa-949b-c12520d80d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:42.944287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 559/1140] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:10:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:10:42.974372 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ffd5a-7bf8-4dc6-9b71-5c86bafcc173 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:43.084624 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:43.109030 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ffd5a-7bf8-4dc6-9b71-5c86bafcc173 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:43.154785 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174d3e66-43cc-40b3-ba21-e28ad81b7765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:43.155721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d3e66-43cc-40b3-ba21-e28ad81b7765 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:43.156007 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d3e66-43cc-40b3-ba21-e28ad81b7765 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:43.156283 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d3e66-43cc-40b3-ba21-e28ad81b7765 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:43.171801 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174d3e66-43cc-40b3-ba21-e28ad81b7765 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:10:43.181377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-174d3e66-43cc-40b3-ba21-e28ad81b7765 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:10:43.181377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:10:43.181377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:10:43.181377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:10:43.181377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:10:43.181377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:10:43.181377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:10:43.181377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:10:43.181377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:10:43.181377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:10:43.181377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:10:43.181377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:10:43.181377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:10:43.181377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:10:43.181377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:10:43.181377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:10:43.182658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:10:43.182658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:10:43.182658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:10:43.182658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:10:43.182658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:10:43.182658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:10:43.182658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:10:43.182658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:10:43.182658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:10:43.182658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:10:43.182658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:10:43.182658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:10:43.182658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:10:43.182658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:10:43.182658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:10:43.182658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:10:43.183771 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:10:43.183771 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:10:43.183771 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:10:43.183771 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:10:43.183771 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:10:43.183771 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:10:43.183771 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:10:43.183771 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:10:43.183771 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:10:43.183771 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:10:43.183771 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:10:43.183771 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:10:43.183771 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 582/1141] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Jul 25 19:10:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:10:43.191227 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df8023b-2c83-4caa-949b-c12520d80d8c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:43.368518 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:43.378788 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df8023b-2c83-4caa-949b-c12520d80d8c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:43.433292 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a9f8ee-67e2-40db-b934-69c511f0b4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:43.434438 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9f8ee-67e2-40db-b934-69c511f0b4fa None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:43.434821 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9f8ee-67e2-40db-b934-69c511f0b4fa None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:43.435201 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9f8ee-67e2-40db-b934-69c511f0b4fa None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:43.436104 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a9f8ee-67e2-40db-b934-69c511f0b4fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:43.436417 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a9f8ee-67e2-40db-b934-69c511f0b4fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:43.437882 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a9f8ee-67e2-40db-b934-69c511f0b4fa None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:43.458683 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 560/1142] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:10:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:43.465297 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174d3e66-43cc-40b3-ba21-e28ad81b7765 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:43.627163 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174d3e66-43cc-40b3-ba21-e28ad81b7765 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:43.688710 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf4d19-bfa1-49d8-88b0-6b59986117a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:43.689709 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf4d19-bfa1-49d8-88b0-6b59986117a9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:43.690036 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf4d19-bfa1-49d8-88b0-6b59986117a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:43.690355 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf4d19-bfa1-49d8-88b0-6b59986117a9 None admin] PATH_INFO: `/v3/users/1d174d3d2c8149b191747d9fc2a238a8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:43.713761 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:43.718923 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baf4d19-bfa1-49d8-88b0-6b59986117a9 None admin] RBAC: Authorizing `identity:get_user(user_id=1d174d3d2c8149b191747d9fc2a238a8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:43.722398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baf4d19-bfa1-49d8-88b0-6b59986117a9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:43.725985 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 583/1143] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:10:43 2022] GET /identity/v3/users/1d174d3d2c8149b191747d9fc2a238a8 => generated 286 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:10:43.733926 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a9f8ee-67e2-40db-b934-69c511f0b4fa None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:43.883140 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a9f8ee-67e2-40db-b934-69c511f0b4fa None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:43.930942 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4342a181-3339-445a-afe6-d66b26342473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:43.931704 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342a181-3339-445a-afe6-d66b26342473 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:43.931990 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342a181-3339-445a-afe6-d66b26342473 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:43.932273 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342a181-3339-445a-afe6-d66b26342473 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:43.972172 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:44.030239 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4342a181-3339-445a-afe6-d66b26342473 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:44.031793 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4342a181-3339-445a-afe6-d66b26342473 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:44.166117 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 561/1144] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:10:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 19:10:44.173059 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf4d19-bfa1-49d8-88b0-6b59986117a9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:44.313698 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf4d19-bfa1-49d8-88b0-6b59986117a9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:44.380959 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70dbc14-a668-43e5-816b-d4a22b4ebe65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:44.382524 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70dbc14-a668-43e5-816b-d4a22b4ebe65 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:44.383599 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70dbc14-a668-43e5-816b-d4a22b4ebe65 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:44.384119 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70dbc14-a668-43e5-816b-d4a22b4ebe65 None admin] PATH_INFO: `/v3/projects/628999d249dd45579fe7fa452f4d80a7` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:44.386541 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70dbc14-a668-43e5-816b-d4a22b4ebe65 None admin] RBAC: Authorizing `identity:get_project(project_id=628999d249dd45579fe7fa452f4d80a7)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:44.389509 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70dbc14-a668-43e5-816b-d4a22b4ebe65 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:44.392951 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 584/1145] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:10:44 2022] GET /identity/v3/projects/628999d249dd45579fe7fa452f4d80a7 => generated 311 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:44.403769 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4342a181-3339-445a-afe6-d66b26342473 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:44.507746 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:44.567055 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4342a181-3339-445a-afe6-d66b26342473 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:44.629237 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a299f639-4c8b-40ed-aba1-1fd59af4de53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:44.629992 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299f639-4c8b-40ed-aba1-1fd59af4de53 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:44.630325 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299f639-4c8b-40ed-aba1-1fd59af4de53 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:44.630646 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299f639-4c8b-40ed-aba1-1fd59af4de53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:44.631620 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a299f639-4c8b-40ed-aba1-1fd59af4de53 None admin] RBAC: Adding query filter params (role.id=ed7fec18968241efa76a0ee769d96da9, scope.project.id=628999d249dd45579fe7fa452f4d80a7, user.id=1d174d3d2c8149b191747d9fc2a238a8) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:44.632074 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a299f639-4c8b-40ed-aba1-1fd59af4de53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:44.634332 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a299f639-4c8b-40ed-aba1-1fd59af4de53 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:44.641412 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 562/1146] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:10:44 2022] GET /identity/v3/role_assignments?user.id=1d174d3d2c8149b191747d9fc2a238a8&scope.project.id=628999d249dd45579fe7fa452f4d80a7&role.id=ed7fec18968241efa76a0ee769d96da9 => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:10:47.200440 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e913d07-7fff-40c4-87af-964dcd980335 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:47.201015 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e913d07-7fff-40c4-87af-964dcd980335 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:47.201440 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e913d07-7fff-40c4-87af-964dcd980335 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:47.204964 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 585/1147] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:10:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:10:47.212982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b7578-de6c-4d66-8270-2d8d46c1cc50 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:47.214138 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b7578-de6c-4d66-8270-2d8d46c1cc50 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:47.214480 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b7578-de6c-4d66-8270-2d8d46c1cc50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:47.236279 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f49b7578-de6c-4d66-8270-2d8d46c1cc50 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:10:47.237636 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f49b7578-de6c-4d66-8270-2d8d46c1cc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:47.333159 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 563/1148] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:10:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:10:47.374508 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e913d07-7fff-40c4-87af-964dcd980335 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:47.529604 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:47.578994 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e913d07-7fff-40c4-87af-964dcd980335 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:47.630926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e2601-8f4b-47f0-b629-fe0700b48650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:47.631842 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e2601-8f4b-47f0-b629-fe0700b48650 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:47.632130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e2601-8f4b-47f0-b629-fe0700b48650 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:47.632411 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e2601-8f4b-47f0-b629-fe0700b48650 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:47.646911 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e2601-8f4b-47f0-b629-fe0700b48650 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:10:47.655048 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-849e2601-8f4b-47f0-b629-fe0700b48650 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:10:47.655048 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:10:47.655048 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:10:47.655048 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:10:47.655048 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:10:47.655048 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:10:47.655048 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:10:47.655048 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:10:47.655048 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:10:47.655048 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:10:47.655048 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:10:47.655048 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:10:47.655048 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:10:47.655048 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:10:47.655048 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:10:47.655048 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:10:47.656312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:10:47.656312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:10:47.656312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:10:47.656312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:10:47.656312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:10:47.656312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:10:47.656312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:10:47.656312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:10:47.656312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:10:47.656312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:10:47.656312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:10:47.656312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:10:47.656312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:10:47.656312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:10:47.656312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:10:47.656312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:10:47.657285 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:10:47.657285 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:10:47.657285 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:10:47.657285 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:10:47.657285 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:10:47.657285 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:10:47.657285 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:10:47.657285 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:10:47.657285 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:10:47.657285 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:10:47.657285 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:10:47.657285 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:10:47.657285 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 586/1149] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Jul 25 19:10:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:10:47.665014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b7578-de6c-4d66-8270-2d8d46c1cc50 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:47.753603 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:47.818316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b7578-de6c-4d66-8270-2d8d46c1cc50 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:47.865425 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112abc92-231c-4f19-8e16-d9fe25a81d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:47.866189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112abc92-231c-4f19-8e16-d9fe25a81d28 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:47.866531 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112abc92-231c-4f19-8e16-d9fe25a81d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:47.866865 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112abc92-231c-4f19-8e16-d9fe25a81d28 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:47.868334 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112abc92-231c-4f19-8e16-d9fe25a81d28 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:47.868334 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112abc92-231c-4f19-8e16-d9fe25a81d28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:47.869959 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112abc92-231c-4f19-8e16-d9fe25a81d28 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:47.880638 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 564/1150] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:10:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:47.889404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e2601-8f4b-47f0-b629-fe0700b48650 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:48.107476 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e2601-8f4b-47f0-b629-fe0700b48650 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:48.163104 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:48.178541 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd8538-16ae-4af8-a26b-298b7cee50bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:48.180701 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd8538-16ae-4af8-a26b-298b7cee50bb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:48.181208 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd8538-16ae-4af8-a26b-298b7cee50bb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:48.181711 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd8538-16ae-4af8-a26b-298b7cee50bb None admin] PATH_INFO: `/v3/users/1d174d3d2c8149b191747d9fc2a238a8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:48.184243 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fd8538-16ae-4af8-a26b-298b7cee50bb None admin] RBAC: Authorizing `identity:get_user(user_id=1d174d3d2c8149b191747d9fc2a238a8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:48.187129 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fd8538-16ae-4af8-a26b-298b7cee50bb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:48.190551 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 587/1151] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:10:47 2022] GET /identity/v3/users/1d174d3d2c8149b191747d9fc2a238a8 => generated 286 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:48.199880 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112abc92-231c-4f19-8e16-d9fe25a81d28 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:48.363715 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112abc92-231c-4f19-8e16-d9fe25a81d28 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:48.381856 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:48.413276 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7372b8b1-d0c0-4382-8b1a-f5bb5453f84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:48.414435 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372b8b1-d0c0-4382-8b1a-f5bb5453f84e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:48.414727 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372b8b1-d0c0-4382-8b1a-f5bb5453f84e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:48.415003 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372b8b1-d0c0-4382-8b1a-f5bb5453f84e None admin] PATH_INFO: `/v3/projects/628999d249dd45579fe7fa452f4d80a7` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:48.416423 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7372b8b1-d0c0-4382-8b1a-f5bb5453f84e None admin] RBAC: Authorizing `identity:get_project(project_id=628999d249dd45579fe7fa452f4d80a7)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:48.417917 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7372b8b1-d0c0-4382-8b1a-f5bb5453f84e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:48.419816 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 565/1152] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:10:48 2022] GET /identity/v3/projects/628999d249dd45579fe7fa452f4d80a7 => generated 311 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:10:48.427624 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd8538-16ae-4af8-a26b-298b7cee50bb None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:48.582624 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd8538-16ae-4af8-a26b-298b7cee50bb None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:48.632920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6113b6cc-1e14-4162-a351-327805ade320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:48.634176 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6113b6cc-1e14-4162-a351-327805ade320 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:48.634345 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6113b6cc-1e14-4162-a351-327805ade320 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:48.634654 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6113b6cc-1e14-4162-a351-327805ade320 None admin] PATH_INFO: `/v3/projects/628999d249dd45579fe7fa452f4d80a7/users/1d174d3d2c8149b191747d9fc2a238a8/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:48.637262 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6113b6cc-1e14-4162-a351-327805ade320 None admin] RBAC: Authorizing `identity:create_grant(project_id=628999d249dd45579fe7fa452f4d80a7, user_id=1d174d3d2c8149b191747d9fc2a238a8, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:48.639041 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6113b6cc-1e14-4162-a351-327805ade320 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:48.664497 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 588/1153] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Jul 25 19:10:48 2022] PUT /identity/v3/projects/628999d249dd45579fe7fa452f4d80a7/users/1d174d3d2c8149b191747d9fc2a238a8/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:10:51.156131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b761b-4aab-483a-a1c5-93c78696449c None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:51.156943 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b761b-4aab-483a-a1c5-93c78696449c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:51.157109 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b761b-4aab-483a-a1c5-93c78696449c None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:51.159556 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 566/1154] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:10:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:10:51.167430 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc1fb3-15f8-48b1-a8e9-127b5a91d605 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:51.168539 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc1fb3-15f8-48b1-a8e9-127b5a91d605 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:51.168755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc1fb3-15f8-48b1-a8e9-127b5a91d605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:51.194519 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-90fc1fb3-15f8-48b1-a8e9-127b5a91d605 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:10:51.196026 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-90fc1fb3-15f8-48b1-a8e9-127b5a91d605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:51.263442 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:51.288695 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 589/1155] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:10:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:10:51.314402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1b761b-4aab-483a-a1c5-93c78696449c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:51.446238 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1b761b-4aab-483a-a1c5-93c78696449c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:51.487400 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513b53d-4bd9-405c-ba13-2fdd992283ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:51.488258 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513b53d-4bd9-405c-ba13-2fdd992283ed None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:51.488611 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513b53d-4bd9-405c-ba13-2fdd992283ed None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:51.488923 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513b53d-4bd9-405c-ba13-2fdd992283ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:51.504027 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c513b53d-4bd9-405c-ba13-2fdd992283ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:10:51.514306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-c513b53d-4bd9-405c-ba13-2fdd992283ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:10:51.514306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:10:51.514306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:10:51.514306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:10:51.514306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:10:51.514306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:10:51.514306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:10:51.514306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:10:51.514306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:10:51.514306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:10:51.514306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:10:51.514306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:10:51.514306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:10:51.514306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:10:51.514306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:10:51.514306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:10:51.525976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:10:51.525976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:10:51.525976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:10:51.525976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:10:51.525976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:10:51.525976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:10:51.525976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:10:51.525976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:10:51.525976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:10:51.525976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:10:51.525976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:10:51.525976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:10:51.525976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:10:51.525976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:10:51.525976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:10:51.525976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:10:51.526925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:10:51.526925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:10:51.526925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:10:51.526925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:10:51.526925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:10:51.526925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:10:51.526925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:10:51.526925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:10:51.526925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:10:51.526925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:10:51.526925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:10:51.526925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:10:51.526925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 567/1156] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Jul 25 19:10:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:10:51.527875 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc1fb3-15f8-48b1-a8e9-127b5a91d605 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:51.678608 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc1fb3-15f8-48b1-a8e9-127b5a91d605 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:51.729757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b110e-3d7e-4508-b5d8-774292a6ad75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:51.730393 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b110e-3d7e-4508-b5d8-774292a6ad75 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:51.730688 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b110e-3d7e-4508-b5d8-774292a6ad75 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:51.730965 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b110e-3d7e-4508-b5d8-774292a6ad75 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:51.731771 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b110e-3d7e-4508-b5d8-774292a6ad75 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:51.732208 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b110e-3d7e-4508-b5d8-774292a6ad75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:51.733746 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b110e-3d7e-4508-b5d8-774292a6ad75 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:51.745558 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 590/1157] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:10:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:51.752228 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513b53d-4bd9-405c-ba13-2fdd992283ed None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:51.830799 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:51.947384 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513b53d-4bd9-405c-ba13-2fdd992283ed None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:52.021981 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170b4f4-a86e-457a-a997-2a9d15f54cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:52.024424 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170b4f4-a86e-457a-a997-2a9d15f54cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:52.025039 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170b4f4-a86e-457a-a997-2a9d15f54cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:52.025516 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170b4f4-a86e-457a-a997-2a9d15f54cc9 None admin] PATH_INFO: `/v3/users/1d174d3d2c8149b191747d9fc2a238a8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:52.028008 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d170b4f4-a86e-457a-a997-2a9d15f54cc9 None admin] RBAC: Authorizing `identity:get_user(user_id=1d174d3d2c8149b191747d9fc2a238a8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:52.030761 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d170b4f4-a86e-457a-a997-2a9d15f54cc9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:52.034111 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 568/1158] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:10:51 2022] GET /identity/v3/users/1d174d3d2c8149b191747d9fc2a238a8 => generated 286 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:52.040637 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b110e-3d7e-4508-b5d8-774292a6ad75 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:52.106035 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:52.170292 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b110e-3d7e-4508-b5d8-774292a6ad75 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:52.213492 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a6636-2b39-49f7-a830-d97b99ef2100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:52.214165 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a6636-2b39-49f7-a830-d97b99ef2100 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:52.214447 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a6636-2b39-49f7-a830-d97b99ef2100 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:52.214719 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a6636-2b39-49f7-a830-d97b99ef2100 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:52.310070 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4a6636-2b39-49f7-a830-d97b99ef2100 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:52.311812 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4a6636-2b39-49f7-a830-d97b99ef2100 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:52.383907 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:52.457251 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 591/1159] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:10:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 19:10:52.467077 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170b4f4-a86e-457a-a997-2a9d15f54cc9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:52.590123 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170b4f4-a86e-457a-a997-2a9d15f54cc9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:52.604595 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:52.635245 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416bc760-3f0c-4932-9915-9b7b29669d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:52.636217 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416bc760-3f0c-4932-9915-9b7b29669d21 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:52.636548 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416bc760-3f0c-4932-9915-9b7b29669d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:52.636873 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416bc760-3f0c-4932-9915-9b7b29669d21 None admin] PATH_INFO: `/v3/projects/628999d249dd45579fe7fa452f4d80a7` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:52.638250 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416bc760-3f0c-4932-9915-9b7b29669d21 None admin] RBAC: Authorizing `identity:get_project(project_id=628999d249dd45579fe7fa452f4d80a7)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:52.640000 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416bc760-3f0c-4932-9915-9b7b29669d21 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:52.641892 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 569/1160] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:10:52 2022] GET /identity/v3/projects/628999d249dd45579fe7fa452f4d80a7 => generated 311 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:10:52.650265 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a6636-2b39-49f7-a830-d97b99ef2100 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:52.775076 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a6636-2b39-49f7-a830-d97b99ef2100 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:52.819267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc9f1c-3ed8-4c8f-9ca1-f10653bea97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:52.820111 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc9f1c-3ed8-4c8f-9ca1-f10653bea97f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:52.820553 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc9f1c-3ed8-4c8f-9ca1-f10653bea97f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:52.821237 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc9f1c-3ed8-4c8f-9ca1-f10653bea97f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:52.821714 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dc9f1c-3ed8-4c8f-9ca1-f10653bea97f None admin] RBAC: Adding query filter params (role.id=ed7fec18968241efa76a0ee769d96da9, scope.project.id=628999d249dd45579fe7fa452f4d80a7, user.id=1d174d3d2c8149b191747d9fc2a238a8) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:52.822045 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dc9f1c-3ed8-4c8f-9ca1-f10653bea97f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:52.823641 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dc9f1c-3ed8-4c8f-9ca1-f10653bea97f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:52.831188 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 592/1161] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:10:52 2022] GET /identity/v3/role_assignments?user.id=1d174d3d2c8149b191747d9fc2a238a8&scope.project.id=628999d249dd45579fe7fa452f4d80a7&role.id=ed7fec18968241efa76a0ee769d96da9 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:10:55.266146 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9068f-69e5-4361-bc7b-d5afe617d630 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:55.266980 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9068f-69e5-4361-bc7b-d5afe617d630 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:55.266980 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9068f-69e5-4361-bc7b-d5afe617d630 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:55.268397 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 570/1162] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:10:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:10:55.277007 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32defb27-aa15-4889-a3c1-68070aa854d5 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:55.277626 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32defb27-aa15-4889-a3c1-68070aa854d5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:55.277626 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32defb27-aa15-4889-a3c1-68070aa854d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:55.299754 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-32defb27-aa15-4889-a3c1-68070aa854d5 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:10:55.301018 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-32defb27-aa15-4889-a3c1-68070aa854d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:55.310482 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:55.389366 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 593/1163] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:10:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 19:10:55.414940 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e9068f-69e5-4361-bc7b-d5afe617d630 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:55.544209 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e9068f-69e5-4361-bc7b-d5afe617d630 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:55.593957 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a137a774-e511-407b-84b2-99353e8e717a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:55.594703 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137a774-e511-407b-84b2-99353e8e717a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:55.595035 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137a774-e511-407b-84b2-99353e8e717a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:55.595356 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137a774-e511-407b-84b2-99353e8e717a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:55.665211 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:55.697163 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a137a774-e511-407b-84b2-99353e8e717a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:55.699092 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a137a774-e511-407b-84b2-99353e8e717a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:55.877300 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 571/1164] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:10:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 19:10:55.886338 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32defb27-aa15-4889-a3c1-68070aa854d5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:56.052775 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32defb27-aa15-4889-a3c1-68070aa854d5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:56.099853 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d335b71-cf06-4292-bbf8-3e5f2005dca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:56.100683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d335b71-cf06-4292-bbf8-3e5f2005dca8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:56.100979 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d335b71-cf06-4292-bbf8-3e5f2005dca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:56.101779 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d335b71-cf06-4292-bbf8-3e5f2005dca8 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:56.110862 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d335b71-cf06-4292-bbf8-3e5f2005dca8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:56.112383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d335b71-cf06-4292-bbf8-3e5f2005dca8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:56.128935 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-9d335b71-cf06-4292-bbf8-3e5f2005dca8 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 25 19:10:56.128935 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:10:56.128935 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 25 19:10:56.128935 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 25 19:10:56.128935 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:10:56.128935 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:10:56.128935 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:10:56.128935 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:10:56.128935 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 25 19:10:56.128935 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:10:56.128935 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 19:10:56.128935 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:10:56.128935 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:10:56.128935 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:10:56.128935 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:10:56.128935 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:10:56.128935 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:10:56.130372 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:10:56.130372 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:10:56.130372 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:10:56.130372 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:10:56.130372 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:10:56.130372 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:10:56.130372 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:10:56.130372 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:10:56.130372 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:10:56.130372 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:10:56.130372 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:10:56.130372 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:10:56.130372 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:10:56.130372 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:10:56.130372 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:10:56.130372 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:10:56.131801 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:10:56.131801 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:10:56.131801 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:10:56.131801 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:10:56.131801 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:10:56.131801 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:10:56.131801 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:10:56.131801 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:10:56.131801 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:10:56.131801 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:10:56.131801 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 25 19:10:56.131801 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:10:56.131801 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 25 19:10:56.131801 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:10:56.131801 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 594/1165] 10.222.0.18 () {56 vars in 1117 bytes} [Mon Jul 25 19:10:55 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:10:56.140483 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a137a774-e511-407b-84b2-99353e8e717a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:56.247991 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:56.285937 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a137a774-e511-407b-84b2-99353e8e717a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:56.357491 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac97702-ab04-4054-b0e3-90114edd2864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:56.358742 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac97702-ab04-4054-b0e3-90114edd2864 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:56.359227 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac97702-ab04-4054-b0e3-90114edd2864 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:56.359741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac97702-ab04-4054-b0e3-90114edd2864 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:56.361021 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac97702-ab04-4054-b0e3-90114edd2864 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:10:56.361564 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac97702-ab04-4054-b0e3-90114edd2864 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:56.364581 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac97702-ab04-4054-b0e3-90114edd2864 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:56.381875 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 572/1166] 10.222.0.18 () {56 vars in 1115 bytes} [Mon Jul 25 19:10:56 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:10:59.102377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84608c62-eb6d-477c-a82e-8d127772820b None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:59.105954 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84608c62-eb6d-477c-a82e-8d127772820b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:59.105954 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84608c62-eb6d-477c-a82e-8d127772820b None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:59.105954 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 595/1167] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:10:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:10:59.112129 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64cb820-96f9-4c0e-b59a-1a2d0f5b6120 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:59.112727 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64cb820-96f9-4c0e-b59a-1a2d0f5b6120 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:59.112727 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64cb820-96f9-4c0e-b59a-1a2d0f5b6120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:59.134633 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c64cb820-96f9-4c0e-b59a-1a2d0f5b6120 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:10:59.135970 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c64cb820-96f9-4c0e-b59a-1a2d0f5b6120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:10:59.239004 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 573/1168] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:10:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:10:59.265003 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84608c62-eb6d-477c-a82e-8d127772820b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:10:59.270513 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:10:59.467038 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84608c62-eb6d-477c-a82e-8d127772820b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:10:59.531037 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda14423-46ce-485b-a07f-2d4764e5fca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:10:59.532275 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda14423-46ce-485b-a07f-2d4764e5fca3 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:10:59.532797 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda14423-46ce-485b-a07f-2d4764e5fca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:10:59.533309 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda14423-46ce-485b-a07f-2d4764e5fca3 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:10:59.534804 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda14423-46ce-485b-a07f-2d4764e5fca3 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:10:59.537587 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda14423-46ce-485b-a07f-2d4764e5fca3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:10:59.567716 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 596/1169] 10.222.0.18 () {60 vars in 1138 bytes} [Mon Jul 25 19:10:59 2022] POST /identity/v3/domains => generated 267 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:11:02.060058 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226aa4b-acbc-4d88-9e4d-550ee74487b6 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:11:02.060798 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226aa4b-acbc-4d88-9e4d-550ee74487b6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:11:02.060798 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226aa4b-acbc-4d88-9e4d-550ee74487b6 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:11:02.062560 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 574/1170] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:11:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:11:02.075354 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f483713-ccf9-4b3e-8d9e-cb96b5705431 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:11:02.076820 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f483713-ccf9-4b3e-8d9e-cb96b5705431 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:11:02.077344 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f483713-ccf9-4b3e-8d9e-cb96b5705431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:11:02.112141 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3f483713-ccf9-4b3e-8d9e-cb96b5705431 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:11:02.114401 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3f483713-ccf9-4b3e-8d9e-cb96b5705431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:11:02.220223 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:11:02.237044 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 597/1171] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:11:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:11:02.267559 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0226aa4b-acbc-4d88-9e4d-550ee74487b6 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:11:02.387981 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:11:02.442030 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0226aa4b-acbc-4d88-9e4d-550ee74487b6 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:11:02.494859 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fffc89c-350d-4240-9500-fb4c01a0dd40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:11:02.495896 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fffc89c-350d-4240-9500-fb4c01a0dd40 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:11:02.496325 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fffc89c-350d-4240-9500-fb4c01a0dd40 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:11:02.496634 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fffc89c-350d-4240-9500-fb4c01a0dd40 None admin] PATH_INFO: `/v3/domains/92e8cef3c66a4c25a346687eefdcc109` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:11:02.515272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fffc89c-350d-4240-9500-fb4c01a0dd40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=92e8cef3c66a4c25a346687eefdcc109)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:11:02.517090 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fffc89c-350d-4240-9500-fb4c01a0dd40 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:11:02.519149 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 575/1172] 10.222.0.18 () {56 vars in 1183 bytes} [Mon Jul 25 19:11:02 2022] GET /identity/v3/domains/92e8cef3c66a4c25a346687eefdcc109 => generated 267 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:11:02.526390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f483713-ccf9-4b3e-8d9e-cb96b5705431 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:11:02.667860 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f483713-ccf9-4b3e-8d9e-cb96b5705431 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:11:02.722443 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a472ca-f8b2-4620-9404-b3672741d2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:11:02.723261 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a472ca-f8b2-4620-9404-b3672741d2ea None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:11:02.723611 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a472ca-f8b2-4620-9404-b3672741d2ea None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:11:02.723993 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a472ca-f8b2-4620-9404-b3672741d2ea None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:11:02.724903 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a472ca-f8b2-4620-9404-b3672741d2ea None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:11:02.727163 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a472ca-f8b2-4620-9404-b3672741d2ea None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:11:02.774748 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 598/1173] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Jul 25 19:11:02 2022] POST /identity/v3/projects => generated 361 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:11:05.166700 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e63227-02e0-46b5-b26b-2eb8a1acddb5 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:11:05.167576 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e63227-02e0-46b5-b26b-2eb8a1acddb5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:11:05.167764 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e63227-02e0-46b5-b26b-2eb8a1acddb5 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:11:05.170484 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 576/1174] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:11:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:11:05.177958 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67307d01-c8f3-45a0-adb8-cfd576d7da31 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:11:05.178566 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67307d01-c8f3-45a0-adb8-cfd576d7da31 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:11:05.178946 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67307d01-c8f3-45a0-adb8-cfd576d7da31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:11:05.213382 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-67307d01-c8f3-45a0-adb8-cfd576d7da31 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:11:05.215460 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-67307d01-c8f3-45a0-adb8-cfd576d7da31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:11:05.302194 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:11:05.338855 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 599/1175] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:11:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:11:05.365415 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e63227-02e0-46b5-b26b-2eb8a1acddb5 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:11:05.517218 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e63227-02e0-46b5-b26b-2eb8a1acddb5 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:11:05.559365 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:11:05.567498 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899ea26a-cd34-4d98-b13b-c61d21e9999b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:11:05.568706 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ea26a-cd34-4d98-b13b-c61d21e9999b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:11:05.569072 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ea26a-cd34-4d98-b13b-c61d21e9999b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:11:05.569461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ea26a-cd34-4d98-b13b-c61d21e9999b None admin] PATH_INFO: `/v3/domains/92e8cef3c66a4c25a346687eefdcc109` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:11:05.574166 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899ea26a-cd34-4d98-b13b-c61d21e9999b None admin] RBAC: Authorizing `identity:get_domain(domain_id=92e8cef3c66a4c25a346687eefdcc109)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:11:05.576458 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899ea26a-cd34-4d98-b13b-c61d21e9999b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:11:05.578853 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 577/1176] 10.222.0.18 () {56 vars in 1183 bytes} [Mon Jul 25 19:11:05 2022] GET /identity/v3/domains/92e8cef3c66a4c25a346687eefdcc109 => generated 267 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:11:05.589619 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67307d01-c8f3-45a0-adb8-cfd576d7da31 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:11:05.792124 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67307d01-c8f3-45a0-adb8-cfd576d7da31 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:11:05.861526 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b253a0b1-9bb6-4811-8a90-8bae73b4df67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:11:05.862718 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253a0b1-9bb6-4811-8a90-8bae73b4df67 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:11:05.863207 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253a0b1-9bb6-4811-8a90-8bae73b4df67 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:11:05.863731 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253a0b1-9bb6-4811-8a90-8bae73b4df67 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:11:05.865102 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b253a0b1-9bb6-4811-8a90-8bae73b4df67 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:11:05.868009 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b253a0b1-9bb6-4811-8a90-8bae73b4df67 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:11:05.902918 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 600/1177] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Jul 25 19:11:05 2022] POST /identity/v3/users => generated 311 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:11:08.610233 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5353b410-8736-44ab-994c-27d0f828668d None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:11:08.611088 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5353b410-8736-44ab-994c-27d0f828668d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:11:08.611088 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5353b410-8736-44ab-994c-27d0f828668d None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:11:08.612901 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 578/1178] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:11:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:11:08.622020 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866a329-a392-497d-ab13-e6ea68cc11ce None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:11:08.622868 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866a329-a392-497d-ab13-e6ea68cc11ce None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:11:08.623028 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866a329-a392-497d-ab13-e6ea68cc11ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:11:08.657555 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5866a329-a392-497d-ab13-e6ea68cc11ce None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:11:08.658811 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5866a329-a392-497d-ab13-e6ea68cc11ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:11:08.690317 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:11:08.756937 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 601/1179] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:11:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:11:08.782579 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5353b410-8736-44ab-994c-27d0f828668d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:11:08.930713 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5353b410-8736-44ab-994c-27d0f828668d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:11:08.975387 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae53638-b1bc-4784-a846-80e022ec8659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:11:08.976895 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae53638-b1bc-4784-a846-80e022ec8659 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:11:08.976895 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae53638-b1bc-4784-a846-80e022ec8659 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:11:08.976895 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae53638-b1bc-4784-a846-80e022ec8659 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:11:08.994526 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae53638-b1bc-4784-a846-80e022ec8659 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:11:09.005716 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-0ae53638-b1bc-4784-a846-80e022ec8659 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:11:09.005716 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:11:09.005716 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:11:09.005716 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:11:09.005716 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:11:09.005716 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:11:09.005716 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:11:09.005716 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:11:09.005716 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:11:09.005716 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:11:09.005716 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:11:09.005716 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:11:09.005716 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:11:09.005716 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:11:09.005716 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:11:09.005716 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:11:09.007982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:11:09.007982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:11:09.007982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:11:09.007982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:11:09.007982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:11:09.007982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:11:09.007982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:11:09.007982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:11:09.007982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:11:09.007982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:11:09.007982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:11:09.007982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:11:09.007982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:11:09.007982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:11:09.007982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:11:09.007982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:11:09.010025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:11:09.010025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:11:09.010025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:11:09.010025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:11:09.010025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:11:09.010025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:11:09.010025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:11:09.010025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:11:09.010025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:11:09.010025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:11:09.010025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:11:09.010025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:11:09.010025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 579/1180] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Jul 25 19:11:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:11:09.016023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5866a329-a392-497d-ab13-e6ea68cc11ce None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:11:09.158201 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5866a329-a392-497d-ab13-e6ea68cc11ce None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:11:09.203887 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b9db2-315e-4ce2-a1d4-d985e2f7011b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:11:09.204451 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b9db2-315e-4ce2-a1d4-d985e2f7011b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:11:09.204735 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b9db2-315e-4ce2-a1d4-d985e2f7011b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:11:09.205010 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b9db2-315e-4ce2-a1d4-d985e2f7011b None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:11:09.205851 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8b9db2-315e-4ce2-a1d4-d985e2f7011b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:11:09.206163 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8b9db2-315e-4ce2-a1d4-d985e2f7011b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:11:09.207640 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8b9db2-315e-4ce2-a1d4-d985e2f7011b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:11:09.218874 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 602/1181] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:11:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:11:09.226771 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae53638-b1bc-4784-a846-80e022ec8659 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:11:09.328103 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:11:09.397312 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae53638-b1bc-4784-a846-80e022ec8659 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:11:09.449460 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db89bf-a8f3-4e72-9e95-d7a4d6cba305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:11:09.451028 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db89bf-a8f3-4e72-9e95-d7a4d6cba305 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:11:09.451437 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db89bf-a8f3-4e72-9e95-d7a4d6cba305 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:11:09.451850 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db89bf-a8f3-4e72-9e95-d7a4d6cba305 None admin] PATH_INFO: `/v3/users/0c1a9f40729c486fb22a762f0ec1bc45` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:11:09.478080 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2db89bf-a8f3-4e72-9e95-d7a4d6cba305 None admin] RBAC: Authorizing `identity:get_user(user_id=0c1a9f40729c486fb22a762f0ec1bc45)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:11:09.480598 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2db89bf-a8f3-4e72-9e95-d7a4d6cba305 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:11:09.484035 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 580/1182] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:11:09 2022] GET /identity/v3/users/0c1a9f40729c486fb22a762f0ec1bc45 => generated 311 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:11:09.493989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b9db2-315e-4ce2-a1d4-d985e2f7011b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:11:09.545561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:11:09.661448 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b9db2-315e-4ce2-a1d4-d985e2f7011b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:11:09.717151 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99575d2a-4594-419b-92e9-2f7a9a7ac046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:11:09.717869 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99575d2a-4594-419b-92e9-2f7a9a7ac046 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:11:09.718207 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99575d2a-4594-419b-92e9-2f7a9a7ac046 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:11:09.718522 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99575d2a-4594-419b-92e9-2f7a9a7ac046 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:11:09.819135 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99575d2a-4594-419b-92e9-2f7a9a7ac046 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:11:09.821856 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99575d2a-4594-419b-92e9-2f7a9a7ac046 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:11:09.873949 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:11:10.026777 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 603/1183] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:11:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 19:11:10.033877 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db89bf-a8f3-4e72-9e95-d7a4d6cba305 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:11:10.205387 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db89bf-a8f3-4e72-9e95-d7a4d6cba305 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:11:10.230831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:11:10.253630 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a9d3a-0556-43a8-aa86-3d6e505eb437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:11:10.254668 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a9d3a-0556-43a8-aa86-3d6e505eb437 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:11:10.254957 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a9d3a-0556-43a8-aa86-3d6e505eb437 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:11:10.255264 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a9d3a-0556-43a8-aa86-3d6e505eb437 None admin] PATH_INFO: `/v3/projects/0961b6526c8246b5bb090ea1035f034d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:11:10.256788 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879a9d3a-0556-43a8-aa86-3d6e505eb437 None admin] RBAC: Authorizing `identity:get_project(project_id=0961b6526c8246b5bb090ea1035f034d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:11:10.259049 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879a9d3a-0556-43a8-aa86-3d6e505eb437 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:11:10.266125 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 581/1184] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:11:10 2022] GET /identity/v3/projects/0961b6526c8246b5bb090ea1035f034d => generated 361 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:11:10.271121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99575d2a-4594-419b-92e9-2f7a9a7ac046 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:11:10.410588 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99575d2a-4594-419b-92e9-2f7a9a7ac046 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:11:10.443141 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:11:10.459617 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80348220-612c-4591-87f9-c3a0a5c3073c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:11:10.460911 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80348220-612c-4591-87f9-c3a0a5c3073c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:11:10.461931 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80348220-612c-4591-87f9-c3a0a5c3073c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:11:10.462270 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80348220-612c-4591-87f9-c3a0a5c3073c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:11:10.463216 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80348220-612c-4591-87f9-c3a0a5c3073c None admin] RBAC: Adding query filter params (role.id=ed7fec18968241efa76a0ee769d96da9, scope.project.id=0961b6526c8246b5bb090ea1035f034d, user.id=0c1a9f40729c486fb22a762f0ec1bc45) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:11:10.463688 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80348220-612c-4591-87f9-c3a0a5c3073c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:11:10.465471 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80348220-612c-4591-87f9-c3a0a5c3073c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:11:10.473243 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 604/1185] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:11:10 2022] GET /identity/v3/role_assignments?user.id=0c1a9f40729c486fb22a762f0ec1bc45&scope.project.id=0961b6526c8246b5bb090ea1035f034d&role.id=ed7fec18968241efa76a0ee769d96da9 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:11:12.668346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df188203-1a71-40aa-9f0b-0b54837d21de None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:11:12.668972 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df188203-1a71-40aa-9f0b-0b54837d21de None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:11:12.668972 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df188203-1a71-40aa-9f0b-0b54837d21de None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:11:12.673585 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 582/1186] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:11:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:11:12.680384 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03d65e-2fdd-4170-a59d-29e8afa842b2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:11:12.681228 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03d65e-2fdd-4170-a59d-29e8afa842b2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:11:12.681228 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03d65e-2fdd-4170-a59d-29e8afa842b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:11:12.705392 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7f03d65e-2fdd-4170-a59d-29e8afa842b2 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:11:12.706646 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7f03d65e-2fdd-4170-a59d-29e8afa842b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:11:12.794507 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 605/1187] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:11:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:11:12.818216 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df188203-1a71-40aa-9f0b-0b54837d21de None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:11:12.955115 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df188203-1a71-40aa-9f0b-0b54837d21de None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:11:13.009153 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d4c68-de0b-4c9d-bb56-f782b9ecce9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:11:13.010001 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d4c68-de0b-4c9d-bb56-f782b9ecce9b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:11:13.010284 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d4c68-de0b-4c9d-bb56-f782b9ecce9b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:11:13.010563 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d4c68-de0b-4c9d-bb56-f782b9ecce9b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:11:13.024810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97d4c68-de0b-4c9d-bb56-f782b9ecce9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:11:13.032693 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b97d4c68-de0b-4c9d-bb56-f782b9ecce9b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:11:13.032693 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:11:13.032693 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:11:13.032693 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:11:13.032693 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:11:13.032693 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:11:13.032693 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:11:13.032693 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:11:13.032693 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:11:13.032693 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:11:13.032693 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:11:13.032693 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:11:13.032693 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:11:13.032693 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:11:13.032693 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:11:13.032693 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:11:13.034606 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:11:13.034606 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:11:13.034606 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:11:13.034606 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:11:13.034606 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:11:13.034606 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:11:13.034606 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:11:13.034606 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:11:13.034606 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:11:13.034606 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:11:13.034606 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:11:13.034606 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:11:13.034606 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:11:13.034606 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:11:13.034606 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:11:13.034606 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:11:13.036671 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:11:13.036671 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:11:13.036671 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:11:13.036671 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:11:13.036671 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:11:13.036671 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:11:13.036671 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:11:13.036671 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:11:13.036671 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:11:13.036671 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:11:13.036671 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:11:13.036671 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:11:13.036671 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 583/1188] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Jul 25 19:11:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:11:13.043342 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f03d65e-2fdd-4170-a59d-29e8afa842b2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:11:13.199311 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f03d65e-2fdd-4170-a59d-29e8afa842b2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:11:13.221898 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:11:13.247252 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eac1fe-fa80-470e-b721-c7cc04d9b5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:11:13.248291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eac1fe-fa80-470e-b721-c7cc04d9b5e9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:11:13.248699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eac1fe-fa80-470e-b721-c7cc04d9b5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:11:13.249105 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eac1fe-fa80-470e-b721-c7cc04d9b5e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:11:13.250337 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eac1fe-fa80-470e-b721-c7cc04d9b5e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:11:13.250959 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eac1fe-fa80-470e-b721-c7cc04d9b5e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:11:13.253590 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eac1fe-fa80-470e-b721-c7cc04d9b5e9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:11:13.267600 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 606/1189] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:11:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:11:13.276232 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d4c68-de0b-4c9d-bb56-f782b9ecce9b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:11:13.337995 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:11:13.440637 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d4c68-de0b-4c9d-bb56-f782b9ecce9b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:11:13.503445 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a034065-43d3-40a9-83fc-43331bc7f642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:11:13.504936 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a034065-43d3-40a9-83fc-43331bc7f642 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:11:13.505418 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a034065-43d3-40a9-83fc-43331bc7f642 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:11:13.505975 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a034065-43d3-40a9-83fc-43331bc7f642 None admin] PATH_INFO: `/v3/users/0c1a9f40729c486fb22a762f0ec1bc45` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:11:13.508299 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a034065-43d3-40a9-83fc-43331bc7f642 None admin] RBAC: Authorizing `identity:get_user(user_id=0c1a9f40729c486fb22a762f0ec1bc45)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:11:13.510942 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a034065-43d3-40a9-83fc-43331bc7f642 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:11:13.513662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 584/1190] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:11:13 2022] GET /identity/v3/users/0c1a9f40729c486fb22a762f0ec1bc45 => generated 311 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:11:13.520214 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eac1fe-fa80-470e-b721-c7cc04d9b5e9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:11:13.654270 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eac1fe-fa80-470e-b721-c7cc04d9b5e9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:11:13.695282 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8796fc2f-58b5-441b-8819-cb91bf5660ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:11:13.696401 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796fc2f-58b5-441b-8819-cb91bf5660ad None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:11:13.696701 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796fc2f-58b5-441b-8819-cb91bf5660ad None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:11:13.696987 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796fc2f-58b5-441b-8819-cb91bf5660ad None admin] PATH_INFO: `/v3/projects/0961b6526c8246b5bb090ea1035f034d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:11:13.698499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8796fc2f-58b5-441b-8819-cb91bf5660ad None admin] RBAC: Authorizing `identity:get_project(project_id=0961b6526c8246b5bb090ea1035f034d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:11:13.700017 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8796fc2f-58b5-441b-8819-cb91bf5660ad None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:11:13.701972 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 607/1191] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:11:13 2022] GET /identity/v3/projects/0961b6526c8246b5bb090ea1035f034d => generated 361 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:11:13.709316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a034065-43d3-40a9-83fc-43331bc7f642 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:11:13.869993 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a034065-43d3-40a9-83fc-43331bc7f642 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:11:13.886034 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:11:13.908767 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62984fe0-00f3-4b00-9590-84a8ce02b246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:11:13.909683 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62984fe0-00f3-4b00-9590-84a8ce02b246 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:11:13.909964 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62984fe0-00f3-4b00-9590-84a8ce02b246 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:11:13.910246 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62984fe0-00f3-4b00-9590-84a8ce02b246 None admin] PATH_INFO: `/v3/projects/0961b6526c8246b5bb090ea1035f034d/users/0c1a9f40729c486fb22a762f0ec1bc45/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:11:13.912493 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62984fe0-00f3-4b00-9590-84a8ce02b246 None admin] RBAC: Authorizing `identity:create_grant(project_id=0961b6526c8246b5bb090ea1035f034d, user_id=0c1a9f40729c486fb22a762f0ec1bc45, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:11:13.914497 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62984fe0-00f3-4b00-9590-84a8ce02b246 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:11:13.929876 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 585/1192] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Jul 25 19:11:13 2022] PUT /identity/v3/projects/0961b6526c8246b5bb090ea1035f034d/users/0c1a9f40729c486fb22a762f0ec1bc45/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:11:16.075078 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff3a4a-0eba-417b-b93c-d605715793e8 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:11:16.075697 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff3a4a-0eba-417b-b93c-d605715793e8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:11:16.075697 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff3a4a-0eba-417b-b93c-d605715793e8 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:11:16.077087 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 608/1193] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:11:16.083053 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a9ca4-566e-4947-91f8-ff3cddb072db None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:11:16.083663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a9ca4-566e-4947-91f8-ff3cddb072db None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:11:16.083663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a9ca4-566e-4947-91f8-ff3cddb072db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:11:16.106545 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c36a9ca4-566e-4947-91f8-ff3cddb072db None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:11:16.107856 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c36a9ca4-566e-4947-91f8-ff3cddb072db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:11:16.189516 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 586/1194] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:11:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:11:16.214861 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ff3a4a-0eba-417b-b93c-d605715793e8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:11:16.270068 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:11:16.336057 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ff3a4a-0eba-417b-b93c-d605715793e8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:11:16.388657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af03bddc-7340-44e3-a34f-e89f458c5ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:11:16.389507 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03bddc-7340-44e3-a34f-e89f458c5ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:11:16.389895 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03bddc-7340-44e3-a34f-e89f458c5ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:11:16.390185 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03bddc-7340-44e3-a34f-e89f458c5ed4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:11:16.408961 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af03bddc-7340-44e3-a34f-e89f458c5ed4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:11:16.423421 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-af03bddc-7340-44e3-a34f-e89f458c5ed4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:11:16.423421 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:11:16.423421 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:11:16.423421 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:11:16.423421 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:11:16.423421 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:11:16.423421 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:11:16.423421 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:11:16.423421 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:11:16.423421 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:11:16.423421 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:11:16.423421 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:11:16.423421 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:11:16.423421 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:11:16.423421 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:11:16.423421 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:11:16.425255 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:11:16.425255 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:11:16.425255 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:11:16.425255 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:11:16.425255 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:11:16.425255 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:11:16.425255 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:11:16.425255 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:11:16.425255 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:11:16.425255 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:11:16.425255 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:11:16.425255 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:11:16.425255 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:11:16.425255 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:11:16.425255 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:11:16.425255 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:11:16.426721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:11:16.426721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:11:16.426721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:11:16.426721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:11:16.426721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:11:16.426721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:11:16.426721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:11:16.426721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:11:16.426721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:11:16.426721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:11:16.426721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:11:16.426721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:11:16.426721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 609/1195] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Jul 25 19:11:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:11:16.436489 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36a9ca4-566e-4947-91f8-ff3cddb072db None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:11:16.557421 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36a9ca4-566e-4947-91f8-ff3cddb072db None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:11:16.560494 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:11:16.599163 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69daa762-1297-4edc-bb46-1e9f1c74e7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:11:16.599775 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69daa762-1297-4edc-bb46-1e9f1c74e7e8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:11:16.600063 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69daa762-1297-4edc-bb46-1e9f1c74e7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:11:16.600340 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69daa762-1297-4edc-bb46-1e9f1c74e7e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:11:16.602186 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69daa762-1297-4edc-bb46-1e9f1c74e7e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:11:16.602505 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69daa762-1297-4edc-bb46-1e9f1c74e7e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:11:16.603945 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69daa762-1297-4edc-bb46-1e9f1c74e7e8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:11:16.613386 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 587/1196] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:11:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:11:16.619531 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af03bddc-7340-44e3-a34f-e89f458c5ed4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:11:16.740947 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af03bddc-7340-44e3-a34f-e89f458c5ed4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:11:16.749897 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:11:16.785497 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf2d5de-e3c9-40fc-9544-d6e99f7c9972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:11:16.786424 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf2d5de-e3c9-40fc-9544-d6e99f7c9972 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:11:16.786708 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf2d5de-e3c9-40fc-9544-d6e99f7c9972 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:11:16.786988 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf2d5de-e3c9-40fc-9544-d6e99f7c9972 None admin] PATH_INFO: `/v3/users/0c1a9f40729c486fb22a762f0ec1bc45` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:11:16.789198 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf2d5de-e3c9-40fc-9544-d6e99f7c9972 None admin] RBAC: Authorizing `identity:get_user(user_id=0c1a9f40729c486fb22a762f0ec1bc45)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:11:16.790713 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf2d5de-e3c9-40fc-9544-d6e99f7c9972 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:11:16.792583 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 610/1197] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Jul 25 19:11:16 2022] GET /identity/v3/users/0c1a9f40729c486fb22a762f0ec1bc45 => generated 311 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:11:16.799861 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69daa762-1297-4edc-bb46-1e9f1c74e7e8 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:11:16.927962 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69daa762-1297-4edc-bb46-1e9f1c74e7e8 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:11:16.971037 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc0792-e08f-454e-acb3-d0387a9f1973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:11:16.972052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc0792-e08f-454e-acb3-d0387a9f1973 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:11:16.972342 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc0792-e08f-454e-acb3-d0387a9f1973 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:11:16.972621 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc0792-e08f-454e-acb3-d0387a9f1973 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:11:17.015762 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:11:17.064413 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cc0792-e08f-454e-acb3-d0387a9f1973 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:11:17.066382 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cc0792-e08f-454e-acb3-d0387a9f1973 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:11:17.234303 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 588/1198] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:11:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 19:11:17.241296 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf2d5de-e3c9-40fc-9544-d6e99f7c9972 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:11:17.371468 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf2d5de-e3c9-40fc-9544-d6e99f7c9972 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:11:17.415353 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ac62fc-9f16-41a0-ad9d-016fadb722cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:11:17.416275 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac62fc-9f16-41a0-ad9d-016fadb722cb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:11:17.416564 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac62fc-9f16-41a0-ad9d-016fadb722cb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:11:17.416847 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac62fc-9f16-41a0-ad9d-016fadb722cb None admin] PATH_INFO: `/v3/projects/0961b6526c8246b5bb090ea1035f034d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:11:17.418176 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ac62fc-9f16-41a0-ad9d-016fadb722cb None admin] RBAC: Authorizing `identity:get_project(project_id=0961b6526c8246b5bb090ea1035f034d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:11:17.419691 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ac62fc-9f16-41a0-ad9d-016fadb722cb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:11:17.421556 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 611/1199] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Jul 25 19:11:17 2022] GET /identity/v3/projects/0961b6526c8246b5bb090ea1035f034d => generated 361 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:11:17.428464 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc0792-e08f-454e-acb3-d0387a9f1973 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:11:17.513337 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:11:17.562037 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc0792-e08f-454e-acb3-d0387a9f1973 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:11:17.614914 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e9fd8-64b9-4629-baa9-a5694d64086e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:11:17.615738 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e9fd8-64b9-4629-baa9-a5694d64086e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:11:17.616685 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e9fd8-64b9-4629-baa9-a5694d64086e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:11:17.617879 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e9fd8-64b9-4629-baa9-a5694d64086e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:11:17.619104 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371e9fd8-64b9-4629-baa9-a5694d64086e None admin] RBAC: Adding query filter params (role.id=ed7fec18968241efa76a0ee769d96da9, scope.project.id=0961b6526c8246b5bb090ea1035f034d, user.id=0c1a9f40729c486fb22a762f0ec1bc45) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:11:17.619577 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371e9fd8-64b9-4629-baa9-a5694d64086e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:11:17.621373 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371e9fd8-64b9-4629-baa9-a5694d64086e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:11:17.631198 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 589/1200] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:11:17 2022] GET /identity/v3/role_assignments?user.id=0c1a9f40729c486fb22a762f0ec1bc45&scope.project.id=0961b6526c8246b5bb090ea1035f034d&role.id=ed7fec18968241efa76a0ee769d96da9 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:11:27.409262 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=79 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:11:27.626936 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=35 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:20.826807 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba87988-6437-4f5c-8e90-98ec90fcf495 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:20.827449 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba87988-6437-4f5c-8e90-98ec90fcf495 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:20.827887 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba87988-6437-4f5c-8e90-98ec90fcf495 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:20.831618 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 612/1201] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:12:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:12:20.839170 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f180916-586d-4236-8a89-c33ea4d406de None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:20.840273 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f180916-586d-4236-8a89-c33ea4d406de None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:20.840439 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f180916-586d-4236-8a89-c33ea4d406de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:20.879433 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1f180916-586d-4236-8a89-c33ea4d406de None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:12:20.882496 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1f180916-586d-4236-8a89-c33ea4d406de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:12:21.038545 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 590/1202] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:12:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:12:21.078003 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba87988-6437-4f5c-8e90-98ec90fcf495 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:21.312191 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba87988-6437-4f5c-8e90-98ec90fcf495 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:21.382974 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9300d-bf51-47c6-ba10-4f7a2498151d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:21.383991 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9300d-bf51-47c6-ba10-4f7a2498151d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:21.384346 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9300d-bf51-47c6-ba10-4f7a2498151d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:21.384604 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9300d-bf51-47c6-ba10-4f7a2498151d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:21.403685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e9300d-bf51-47c6-ba10-4f7a2498151d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:21.406026 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e9300d-bf51-47c6-ba10-4f7a2498151d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:21.438204 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-03e9300d-bf51-47c6-ba10-4f7a2498151d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:12:21.438204 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:12:21.438204 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:12:21.438204 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:12:21.438204 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:12:21.438204 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:12:21.438204 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:12:21.438204 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:12:21.438204 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:12:21.438204 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:12:21.438204 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:12:21.438204 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:12:21.438204 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:12:21.438204 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:12:21.438204 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:12:21.440491 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:12:21.440491 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:12:21.440491 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:12:21.440491 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:12:21.440491 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:12:21.440491 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:12:21.440491 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:12:21.440491 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:12:21.440491 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:12:21.440491 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:12:21.440491 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:12:21.440491 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:12:21.440491 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:12:21.440491 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:12:21.440491 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:12:21.440491 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:12:21.440491 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:12:21.442541 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:12:21.442541 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:12:21.442541 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:12:21.442541 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:12:21.442541 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 613/1203] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:12:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:12:21.450752 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f180916-586d-4236-8a89-c33ea4d406de None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:21.684141 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f180916-586d-4236-8a89-c33ea4d406de None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:21.743983 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:21.751254 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe71733-f409-4a2a-a1cf-9b581f12e01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:21.752386 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe71733-f409-4a2a-a1cf-9b581f12e01c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:21.753002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe71733-f409-4a2a-a1cf-9b581f12e01c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:21.753551 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe71733-f409-4a2a-a1cf-9b581f12e01c None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:21.755259 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe71733-f409-4a2a-a1cf-9b581f12e01c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:12:21.755780 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe71733-f409-4a2a-a1cf-9b581f12e01c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:21.757620 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe71733-f409-4a2a-a1cf-9b581f12e01c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:21.779079 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 591/1204] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:12:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:12:21.789701 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9300d-bf51-47c6-ba10-4f7a2498151d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:21.893674 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:22.003803 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9300d-bf51-47c6-ba10-4f7a2498151d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:22.071613 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ff7f4-7d74-41d5-b635-9bb9d08818f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:22.073361 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ff7f4-7d74-41d5-b635-9bb9d08818f2 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:22.075555 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ff7f4-7d74-41d5-b635-9bb9d08818f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:22.076226 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ff7f4-7d74-41d5-b635-9bb9d08818f2 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:22.077822 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47ff7f4-7d74-41d5-b635-9bb9d08818f2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:22.086697 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47ff7f4-7d74-41d5-b635-9bb9d08818f2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:22.122979 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 614/1205] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Jul 25 19:12:21 2022] POST /identity/v3/users => generated 251 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:12:24.788555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c3a34-046d-43d0-a95d-2fe0d1ffc527 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:24.789168 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c3a34-046d-43d0-a95d-2fe0d1ffc527 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:24.789832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c3a34-046d-43d0-a95d-2fe0d1ffc527 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:24.792614 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 592/1206] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:12:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:12:24.800501 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c7a20-9d27-422f-9f52-475a737e1ff7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:24.801082 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c7a20-9d27-422f-9f52-475a737e1ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:24.801082 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c7a20-9d27-422f-9f52-475a737e1ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:24.834060 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4b9c7a20-9d27-422f-9f52-475a737e1ff7 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:12:24.836515 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4b9c7a20-9d27-422f-9f52-475a737e1ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:12:25.003581 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 615/1207] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:12:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:12:25.029368 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c3a34-046d-43d0-a95d-2fe0d1ffc527 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:25.225856 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c3a34-046d-43d0-a95d-2fe0d1ffc527 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:25.287096 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bec931-aba5-44f9-8e8c-8ad833fbe5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:25.288287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bec931-aba5-44f9-8e8c-8ad833fbe5a2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:25.288856 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bec931-aba5-44f9-8e8c-8ad833fbe5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:25.289261 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bec931-aba5-44f9-8e8c-8ad833fbe5a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:25.312528 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bec931-aba5-44f9-8e8c-8ad833fbe5a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:12:25.325011 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-85bec931-aba5-44f9-8e8c-8ad833fbe5a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:12:25.325011 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:12:25.325011 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:12:25.325011 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:12:25.325011 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:12:25.325011 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:12:25.325011 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:12:25.325011 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:12:25.325011 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:12:25.325011 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:12:25.325011 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:12:25.325011 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:12:25.325011 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:12:25.325011 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:12:25.325011 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:12:25.325011 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:12:25.326266 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:12:25.326266 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:12:25.326266 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:12:25.326266 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:12:25.326266 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:12:25.326266 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:12:25.326266 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:12:25.326266 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:12:25.326266 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:12:25.326266 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:12:25.326266 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:12:25.326266 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:12:25.326266 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:12:25.326266 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:12:25.326266 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:12:25.326266 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:12:25.327272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:12:25.327272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:12:25.327272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:12:25.327272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:12:25.327272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:12:25.327272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:12:25.327272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:12:25.327272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:12:25.327272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:12:25.327272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:12:25.327272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:12:25.327272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:12:25.328797 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 593/1208] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Jul 25 19:12:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:12:25.335483 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c7a20-9d27-422f-9f52-475a737e1ff7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:25.404898 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:25.500208 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c7a20-9d27-422f-9f52-475a737e1ff7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:25.581072 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b19992-52bc-42ab-b04e-47678592ec66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:25.581913 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b19992-52bc-42ab-b04e-47678592ec66 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:25.582248 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b19992-52bc-42ab-b04e-47678592ec66 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:25.583006 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b19992-52bc-42ab-b04e-47678592ec66 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:25.584121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b19992-52bc-42ab-b04e-47678592ec66 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:12:25.584602 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b19992-52bc-42ab-b04e-47678592ec66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:25.585992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b19992-52bc-42ab-b04e-47678592ec66 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:25.602693 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 616/1209] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Jul 25 19:12:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:12:25.605008 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bec931-aba5-44f9-8e8c-8ad833fbe5a2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:25.689109 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:25.765658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bec931-aba5-44f9-8e8c-8ad833fbe5a2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:25.821201 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5bb0a2-3d88-4563-b94f-ce388a8a0e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:25.822517 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bb0a2-3d88-4563-b94f-ce388a8a0e3d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:25.823014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bb0a2-3d88-4563-b94f-ce388a8a0e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:25.823476 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5bb0a2-3d88-4563-b94f-ce388a8a0e3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:25.839294 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5bb0a2-3d88-4563-b94f-ce388a8a0e3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:25.841300 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5bb0a2-3d88-4563-b94f-ce388a8a0e3d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:25.871375 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-4d5bb0a2-3d88-4563-b94f-ce388a8a0e3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:12:25.871375 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:12:25.871375 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:12:25.871375 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:12:25.871375 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:12:25.871375 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:12:25.871375 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:12:25.871375 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:12:25.871375 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:12:25.871375 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:12:25.871375 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:12:25.871375 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:12:25.871375 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:12:25.871375 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:12:25.871375 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:12:25.879107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:12:25.879107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:12:25.879107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:12:25.879107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:12:25.879107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:12:25.879107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:12:25.879107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:12:25.879107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:12:25.879107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:12:25.879107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:12:25.879107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:12:25.879107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:12:25.879107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:12:25.879107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:12:25.879107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:12:25.879107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:12:25.879107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:12:25.880685 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:12:25.880685 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:12:25.880685 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:12:25.880685 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:12:25.880685 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 594/1210] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:12:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:12:25.881400 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b19992-52bc-42ab-b04e-47678592ec66 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:26.049758 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:26.054099 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b19992-52bc-42ab-b04e-47678592ec66 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:26.104975 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2bcf39-8eab-4c01-a528-3d0de582dcc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:26.106187 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2bcf39-8eab-4c01-a528-3d0de582dcc5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:26.106706 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2bcf39-8eab-4c01-a528-3d0de582dcc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:26.107307 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2bcf39-8eab-4c01-a528-3d0de582dcc5 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:26.108900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2bcf39-8eab-4c01-a528-3d0de582dcc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:12:26.109479 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2bcf39-8eab-4c01-a528-3d0de582dcc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:26.113177 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2bcf39-8eab-4c01-a528-3d0de582dcc5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:26.133654 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 617/1211] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:12:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:12:26.141708 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5bb0a2-3d88-4563-b94f-ce388a8a0e3d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:26.295379 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5bb0a2-3d88-4563-b94f-ce388a8a0e3d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:26.297657 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:26.352343 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632d829-4fe3-4902-b48b-1afa716521df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:26.353659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632d829-4fe3-4902-b48b-1afa716521df None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:26.354058 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632d829-4fe3-4902-b48b-1afa716521df None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:26.354472 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632d829-4fe3-4902-b48b-1afa716521df None admin] PATH_INFO: `/v3/users/placement` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:26.368367 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6632d829-4fe3-4902-b48b-1afa716521df None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:26.371260 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6632d829-4fe3-4902-b48b-1afa716521df None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:26.394143 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-6632d829-4fe3-4902-b48b-1afa716521df None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 19:12:26.394143 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:12:26.394143 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:12:26.394143 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:12:26.394143 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:12:26.394143 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:12:26.394143 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:12:26.394143 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:12:26.394143 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:12:26.394143 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:12:26.394143 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:12:26.394143 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:12:26.394143 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:12:26.394143 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:12:26.394143 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:12:26.394143 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:12:26.397168 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:12:26.397168 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:12:26.397168 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:12:26.397168 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:12:26.397168 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:12:26.397168 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:12:26.397168 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:12:26.397168 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:12:26.397168 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:12:26.397168 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:12:26.397168 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:12:26.397168 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:12:26.397168 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:12:26.397168 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:12:26.397168 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:12:26.397168 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:12:26.397168 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:12:26.397168 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:12:26.399192 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:12:26.399192 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:12:26.399192 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:12:26.399192 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:12:26.399192 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:12:26.399192 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:12:26.399192 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:12:26.399192 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:12:26.399192 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 19:12:26.399192 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:12:26.399192 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 595/1212] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:12:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:12:26.406379 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2bcf39-8eab-4c01-a528-3d0de582dcc5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:26.592350 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2bcf39-8eab-4c01-a528-3d0de582dcc5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:26.675913 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d145d7b-6627-4964-bbb2-5fd5e710ea1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:26.676420 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d145d7b-6627-4964-bbb2-5fd5e710ea1c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:26.677146 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d145d7b-6627-4964-bbb2-5fd5e710ea1c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:26.677845 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d145d7b-6627-4964-bbb2-5fd5e710ea1c None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:26.679792 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d145d7b-6627-4964-bbb2-5fd5e710ea1c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:12:26.680360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d145d7b-6627-4964-bbb2-5fd5e710ea1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:26.683152 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d145d7b-6627-4964-bbb2-5fd5e710ea1c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:26.712394 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 618/1213] 10.222.0.18 () {56 vars in 1143 bytes} [Mon Jul 25 19:12:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:12:26.720873 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632d829-4fe3-4902-b48b-1afa716521df None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:26.903310 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632d829-4fe3-4902-b48b-1afa716521df None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:26.984729 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a8d1ae-eddf-49a6-aacc-ab8dfafeddcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:26.985846 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8d1ae-eddf-49a6-aacc-ab8dfafeddcd None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:26.986356 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8d1ae-eddf-49a6-aacc-ab8dfafeddcd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:26.986865 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8d1ae-eddf-49a6-aacc-ab8dfafeddcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:27.045421 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:27.117069 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a8d1ae-eddf-49a6-aacc-ab8dfafeddcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:27.118618 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a8d1ae-eddf-49a6-aacc-ab8dfafeddcd None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:27.328021 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 596/1214] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:12:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 19:12:27.336488 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d145d7b-6627-4964-bbb2-5fd5e710ea1c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:27.370058 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:27.534805 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d145d7b-6627-4964-bbb2-5fd5e710ea1c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:27.624436 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467772a-228f-4560-9ebd-9d4c8cc9315f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:27.626233 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467772a-228f-4560-9ebd-9d4c8cc9315f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:27.626714 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467772a-228f-4560-9ebd-9d4c8cc9315f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:27.627313 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467772a-228f-4560-9ebd-9d4c8cc9315f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:27.652829 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e467772a-228f-4560-9ebd-9d4c8cc9315f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:27.654458 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e467772a-228f-4560-9ebd-9d4c8cc9315f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:27.678120 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e467772a-228f-4560-9ebd-9d4c8cc9315f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:12:27.678120 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:12:27.678120 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:12:27.678120 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:12:27.678120 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:12:27.678120 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:12:27.678120 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:12:27.678120 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:12:27.678120 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:12:27.678120 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:12:27.678120 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:12:27.678120 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:12:27.678120 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:12:27.678120 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:12:27.678120 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:12:27.679136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:12:27.679136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:12:27.679136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:12:27.679136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:12:27.679136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:12:27.679136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:12:27.679136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:12:27.679136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:12:27.679136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:12:27.679136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:12:27.679136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:12:27.679136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:12:27.679136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:12:27.679136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:12:27.679136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:12:27.679136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:12:27.679136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:12:27.680174 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:12:27.680174 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:12:27.680174 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:12:27.680174 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:12:27.680174 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 619/1215] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:12:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:12:27.686855 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a8d1ae-eddf-49a6-aacc-ab8dfafeddcd None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:27.791108 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:27.848727 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a8d1ae-eddf-49a6-aacc-ab8dfafeddcd None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:27.895122 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797eb65c-a645-4ec0-b8fc-275bb8b5e846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:27.895883 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797eb65c-a645-4ec0-b8fc-275bb8b5e846 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:27.896297 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797eb65c-a645-4ec0-b8fc-275bb8b5e846 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:27.896686 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797eb65c-a645-4ec0-b8fc-275bb8b5e846 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:27.897681 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797eb65c-a645-4ec0-b8fc-275bb8b5e846 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:12:27.898048 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797eb65c-a645-4ec0-b8fc-275bb8b5e846 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:27.899834 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797eb65c-a645-4ec0-b8fc-275bb8b5e846 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:27.925616 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 597/1216] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:12:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:12:27.938455 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467772a-228f-4560-9ebd-9d4c8cc9315f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:28.087213 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:28.153721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467772a-228f-4560-9ebd-9d4c8cc9315f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:28.236466 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05007c-14e6-457e-8010-ae8ad47d7c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:28.237911 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05007c-14e6-457e-8010-ae8ad47d7c72 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:28.238587 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05007c-14e6-457e-8010-ae8ad47d7c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:28.240534 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05007c-14e6-457e-8010-ae8ad47d7c72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:28.257390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05007c-14e6-457e-8010-ae8ad47d7c72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:28.260750 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05007c-14e6-457e-8010-ae8ad47d7c72 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:28.298678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-ac05007c-14e6-457e-8010-ae8ad47d7c72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:12:28.298678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:12:28.298678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:12:28.298678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:12:28.298678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:12:28.298678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:12:28.298678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:12:28.298678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:12:28.298678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:12:28.298678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:12:28.298678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:12:28.298678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:12:28.298678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:12:28.298678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:12:28.298678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:12:28.299848 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:12:28.299848 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:12:28.299848 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:12:28.299848 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:12:28.299848 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:12:28.299848 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:12:28.299848 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:12:28.299848 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:12:28.299848 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:12:28.299848 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:12:28.299848 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:12:28.299848 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:12:28.299848 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:12:28.299848 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:12:28.299848 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:12:28.299848 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:12:28.299848 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:12:28.300906 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:12:28.300906 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:12:28.300906 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:12:28.300906 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:12:28.300906 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:12:28.300906 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:12:28.300906 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:12:28.300906 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:12:28.302109 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 620/1217] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:12:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:12:28.314195 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797eb65c-a645-4ec0-b8fc-275bb8b5e846 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:28.503551 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797eb65c-a645-4ec0-b8fc-275bb8b5e846 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:28.547170 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:28.567210 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775cf09-8102-4b0d-b24e-4cf09b3495ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:28.568086 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775cf09-8102-4b0d-b24e-4cf09b3495ed None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:28.568602 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775cf09-8102-4b0d-b24e-4cf09b3495ed None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:28.569142 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775cf09-8102-4b0d-b24e-4cf09b3495ed None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:28.570417 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d775cf09-8102-4b0d-b24e-4cf09b3495ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:12:28.570742 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d775cf09-8102-4b0d-b24e-4cf09b3495ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:28.572907 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d775cf09-8102-4b0d-b24e-4cf09b3495ed None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:28.597045 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 598/1218] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Jul 25 19:12:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:12:28.605401 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05007c-14e6-457e-8010-ae8ad47d7c72 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:28.808485 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05007c-14e6-457e-8010-ae8ad47d7c72 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:28.843354 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:28.851218 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62d938-9963-4ce2-a5d0-ca1e7779709a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:28.852137 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62d938-9963-4ce2-a5d0-ca1e7779709a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:28.852428 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62d938-9963-4ce2-a5d0-ca1e7779709a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:28.852715 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62d938-9963-4ce2-a5d0-ca1e7779709a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:28.853605 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb62d938-9963-4ce2-a5d0-ca1e7779709a None admin] RBAC: Adding query filter params (role.id=23718d83b0864b16b0071e8ffe0b5861, scope.project.id=191069caccd04233b012585e29cfa18b, user.id=0422d6695bf14925bf0f993cb16e64ca) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:12:28.853927 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb62d938-9963-4ce2-a5d0-ca1e7779709a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:28.855637 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb62d938-9963-4ce2-a5d0-ca1e7779709a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:28.865941 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 621/1219] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:12:28 2022] GET /identity/v3/role_assignments?user.id=0422d6695bf14925bf0f993cb16e64ca&scope.project.id=191069caccd04233b012585e29cfa18b&role.id=23718d83b0864b16b0071e8ffe0b5861 => generated 259 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:12:31.904819 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc50e5e-c416-4537-8964-2db5e156501a None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:31.905653 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc50e5e-c416-4537-8964-2db5e156501a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:31.905653 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc50e5e-c416-4537-8964-2db5e156501a None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:31.907380 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 599/1220] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:12:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:12:31.915712 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557537d4-26c6-47b8-9f47-c4e99a1bd6f1 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:31.916435 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557537d4-26c6-47b8-9f47-c4e99a1bd6f1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:31.916435 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557537d4-26c6-47b8-9f47-c4e99a1bd6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:31.943808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-557537d4-26c6-47b8-9f47-c4e99a1bd6f1 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:12:31.945687 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-557537d4-26c6-47b8-9f47-c4e99a1bd6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:12:32.053077 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 622/1221] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:12:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 19:12:32.083838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc50e5e-c416-4537-8964-2db5e156501a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:32.276950 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc50e5e-c416-4537-8964-2db5e156501a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:32.347682 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3d8c0-a582-4e7a-a719-08fcc2e71dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:32.349692 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3d8c0-a582-4e7a-a719-08fcc2e71dec None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:32.350004 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3d8c0-a582-4e7a-a719-08fcc2e71dec None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:32.350494 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3d8c0-a582-4e7a-a719-08fcc2e71dec None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:32.370425 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d3d8c0-a582-4e7a-a719-08fcc2e71dec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:12:32.383316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-33d3d8c0-a582-4e7a-a719-08fcc2e71dec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:12:32.383316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:12:32.383316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:12:32.383316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:12:32.383316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:12:32.383316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:12:32.383316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:12:32.383316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:12:32.383316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:12:32.383316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:12:32.383316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:12:32.383316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:12:32.383316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:12:32.383316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:12:32.383316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:12:32.383316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:12:32.385539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:12:32.385539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:12:32.385539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:12:32.385539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:12:32.385539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:12:32.385539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:12:32.385539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:12:32.385539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:12:32.385539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:12:32.385539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:12:32.385539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:12:32.385539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:12:32.385539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:12:32.385539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:12:32.385539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:12:32.385539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:12:32.387779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:12:32.387779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:12:32.387779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:12:32.387779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:12:32.387779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:12:32.387779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:12:32.387779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:12:32.387779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:12:32.387779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:12:32.387779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:12:32.387779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:12:32.387779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:12:32.390618 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 600/1222] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Jul 25 19:12:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:12:32.391886 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557537d4-26c6-47b8-9f47-c4e99a1bd6f1 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:32.594764 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557537d4-26c6-47b8-9f47-c4e99a1bd6f1 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:32.649173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:32.667017 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b96ec-d202-4b9d-b2fa-d26bd49fb510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:32.667792 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b96ec-d202-4b9d-b2fa-d26bd49fb510 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:32.668083 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b96ec-d202-4b9d-b2fa-d26bd49fb510 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:32.668354 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b96ec-d202-4b9d-b2fa-d26bd49fb510 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:32.669173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80b96ec-d202-4b9d-b2fa-d26bd49fb510 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:12:32.669515 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80b96ec-d202-4b9d-b2fa-d26bd49fb510 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:32.671025 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80b96ec-d202-4b9d-b2fa-d26bd49fb510 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:32.684317 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 623/1223] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Jul 25 19:12:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:12:32.692650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3d8c0-a582-4e7a-a719-08fcc2e71dec None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:32.768917 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:32.889006 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3d8c0-a582-4e7a-a719-08fcc2e71dec None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:32.963211 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8ba9d-8097-4a0d-b9bf-d5934af095b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:32.964499 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8ba9d-8097-4a0d-b9bf-d5934af095b5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:32.964967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8ba9d-8097-4a0d-b9bf-d5934af095b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:32.965408 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8ba9d-8097-4a0d-b9bf-d5934af095b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:32.983339 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c8ba9d-8097-4a0d-b9bf-d5934af095b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:32.985648 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c8ba9d-8097-4a0d-b9bf-d5934af095b5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:33.021428 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-37c8ba9d-8097-4a0d-b9bf-d5934af095b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:12:33.021428 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:12:33.021428 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:12:33.021428 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:12:33.021428 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:12:33.021428 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:12:33.021428 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:12:33.021428 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:12:33.021428 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:12:33.021428 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:12:33.021428 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:12:33.021428 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:12:33.021428 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:12:33.021428 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:12:33.021428 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:12:33.023366 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:12:33.023366 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:12:33.023366 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:12:33.023366 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:12:33.023366 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:12:33.023366 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:12:33.023366 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:12:33.023366 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:12:33.023366 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:12:33.023366 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:12:33.023366 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:12:33.023366 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:12:33.023366 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:12:33.023366 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:12:33.023366 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:12:33.023366 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:12:33.023366 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:12:33.024603 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:12:33.024603 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:12:33.024603 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:12:33.024603 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:12:33.024603 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 601/1224] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:12:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:12:33.032927 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b96ec-d202-4b9d-b2fa-d26bd49fb510 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:33.195026 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b96ec-d202-4b9d-b2fa-d26bd49fb510 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:33.247173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2a599-3549-4991-8893-a58eace3daf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:33.248492 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2a599-3549-4991-8893-a58eace3daf3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:33.249006 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2a599-3549-4991-8893-a58eace3daf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:33.249495 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2a599-3549-4991-8893-a58eace3daf3 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:33.250865 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2a599-3549-4991-8893-a58eace3daf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:12:33.251412 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2a599-3549-4991-8893-a58eace3daf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:33.255967 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2a599-3549-4991-8893-a58eace3daf3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:33.274456 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 624/1225] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:12:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:12:33.284012 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8ba9d-8097-4a0d-b9bf-d5934af095b5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:33.465613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:33.477157 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8ba9d-8097-4a0d-b9bf-d5934af095b5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:33.551782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d6d06-be12-480e-a91d-e810f6aa9bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:33.553962 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d6d06-be12-480e-a91d-e810f6aa9bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:33.554528 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d6d06-be12-480e-a91d-e810f6aa9bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:33.555090 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d6d06-be12-480e-a91d-e810f6aa9bf6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:33.574845 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d6d06-be12-480e-a91d-e810f6aa9bf6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:33.580526 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d6d06-be12-480e-a91d-e810f6aa9bf6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:33.622200 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-db7d6d06-be12-480e-a91d-e810f6aa9bf6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 19:12:33.622200 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:12:33.622200 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:12:33.622200 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:12:33.622200 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:12:33.622200 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:12:33.622200 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:12:33.622200 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:12:33.622200 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:12:33.622200 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:12:33.622200 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:12:33.622200 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:12:33.622200 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:12:33.622200 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:12:33.622200 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:12:33.622200 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:12:33.623481 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:12:33.623481 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:12:33.623481 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:12:33.623481 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:12:33.623481 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:12:33.623481 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:12:33.623481 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:12:33.623481 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:12:33.623481 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:12:33.623481 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:12:33.623481 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:12:33.623481 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:12:33.623481 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:12:33.623481 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:12:33.623481 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:12:33.623481 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:12:33.623481 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:12:33.623481 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:12:33.624710 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:12:33.624710 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:12:33.624710 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:12:33.624710 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:12:33.624710 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:12:33.624710 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:12:33.624710 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:12:33.624710 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:12:33.624710 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 19:12:33.624710 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:12:33.625349 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 602/1226] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:12:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:12:33.637863 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2a599-3549-4991-8893-a58eace3daf3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:33.728385 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:33.819706 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2a599-3549-4991-8893-a58eace3daf3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:33.875872 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a290262-a297-4268-abcc-5317bde9d3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:33.877079 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a290262-a297-4268-abcc-5317bde9d3b6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:33.877374 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a290262-a297-4268-abcc-5317bde9d3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:33.877654 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a290262-a297-4268-abcc-5317bde9d3b6 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:33.878543 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a290262-a297-4268-abcc-5317bde9d3b6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:12:33.878857 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a290262-a297-4268-abcc-5317bde9d3b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:33.881382 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a290262-a297-4268-abcc-5317bde9d3b6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:33.904552 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 625/1227] 10.222.0.18 () {56 vars in 1143 bytes} [Mon Jul 25 19:12:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:12:33.911310 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d6d06-be12-480e-a91d-e810f6aa9bf6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:34.091239 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d6d06-be12-480e-a91d-e810f6aa9bf6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:34.149606 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2004ae-e2b5-49f5-b720-c1cf2d806335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:34.151044 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2004ae-e2b5-49f5-b720-c1cf2d806335 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:34.151600 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2004ae-e2b5-49f5-b720-c1cf2d806335 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:34.152084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2004ae-e2b5-49f5-b720-c1cf2d806335 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:34.174255 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2004ae-e2b5-49f5-b720-c1cf2d806335 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:34.177761 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2004ae-e2b5-49f5-b720-c1cf2d806335 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:34.213967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-2a2004ae-e2b5-49f5-b720-c1cf2d806335 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:12:34.213967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:12:34.213967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:12:34.213967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:12:34.213967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:12:34.213967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:12:34.213967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:12:34.213967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:12:34.213967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:12:34.213967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:12:34.213967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:12:34.213967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:12:34.213967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:12:34.213967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:12:34.213967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:12:34.216362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:12:34.216362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:12:34.216362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:12:34.216362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:12:34.216362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:12:34.216362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:12:34.216362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:12:34.216362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:12:34.216362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:12:34.216362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:12:34.216362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:12:34.216362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:12:34.216362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:12:34.216362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:12:34.216362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:12:34.216362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:12:34.216362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:12:34.218045 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:12:34.218045 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:12:34.218045 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:12:34.218045 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:12:34.218045 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 603/1228] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:12:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:12:34.226794 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a290262-a297-4268-abcc-5317bde9d3b6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:34.370830 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a290262-a297-4268-abcc-5317bde9d3b6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:34.374048 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:34.418365 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5634b516-781a-431a-9a83-6cfa9383874c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:34.419143 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634b516-781a-431a-9a83-6cfa9383874c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:34.419628 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634b516-781a-431a-9a83-6cfa9383874c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:34.420099 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634b516-781a-431a-9a83-6cfa9383874c None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:34.421047 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5634b516-781a-431a-9a83-6cfa9383874c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:12:34.421497 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5634b516-781a-431a-9a83-6cfa9383874c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:34.423723 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5634b516-781a-431a-9a83-6cfa9383874c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:34.439908 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 626/1229] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:12:34.446874 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2004ae-e2b5-49f5-b720-c1cf2d806335 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:34.462347 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:34.657682 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2004ae-e2b5-49f5-b720-c1cf2d806335 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:34.707723 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c2ea9-50b0-43bb-a22e-baeb38f4ec1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:34.708829 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c2ea9-50b0-43bb-a22e-baeb38f4ec1f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:34.709119 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c2ea9-50b0-43bb-a22e-baeb38f4ec1f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:34.709479 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c2ea9-50b0-43bb-a22e-baeb38f4ec1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:34.719788 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568c2ea9-50b0-43bb-a22e-baeb38f4ec1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:34.721833 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568c2ea9-50b0-43bb-a22e-baeb38f4ec1f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:34.741765 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-568c2ea9-50b0-43bb-a22e-baeb38f4ec1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:12:34.741765 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:12:34.741765 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:12:34.741765 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:12:34.741765 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:12:34.741765 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:12:34.741765 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:12:34.741765 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:12:34.741765 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:12:34.741765 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:12:34.741765 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:12:34.741765 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:12:34.741765 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:12:34.741765 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:12:34.741765 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:12:34.743662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:12:34.743662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:12:34.743662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:12:34.743662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:12:34.743662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:12:34.743662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:12:34.743662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:12:34.743662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:12:34.743662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:12:34.743662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:12:34.743662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:12:34.743662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:12:34.743662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:12:34.743662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:12:34.743662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:12:34.743662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:12:34.743662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:12:34.745421 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:12:34.745421 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:12:34.745421 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:12:34.745421 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:12:34.745421 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:12:34.745421 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:12:34.745421 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:12:34.745421 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:12:34.745421 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 604/1230] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:12:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:12:34.751295 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5634b516-781a-431a-9a83-6cfa9383874c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:34.909839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5634b516-781a-431a-9a83-6cfa9383874c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:34.974179 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0885bc9-7d2b-436a-b43b-a8b38e9e32bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:34.975371 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0885bc9-7d2b-436a-b43b-a8b38e9e32bf None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:34.975954 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0885bc9-7d2b-436a-b43b-a8b38e9e32bf None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:34.976436 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0885bc9-7d2b-436a-b43b-a8b38e9e32bf None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:34.977929 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0885bc9-7d2b-436a-b43b-a8b38e9e32bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:12:34.978497 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0885bc9-7d2b-436a-b43b-a8b38e9e32bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:34.981441 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0885bc9-7d2b-436a-b43b-a8b38e9e32bf None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:35.008218 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 627/1231] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Jul 25 19:12:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:12:35.016496 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c2ea9-50b0-43bb-a22e-baeb38f4ec1f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:35.146988 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:35.199052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c2ea9-50b0-43bb-a22e-baeb38f4ec1f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:35.247851 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119df92c-d272-49d2-8ad5-9c64363f80fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:35.248971 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119df92c-d272-49d2-8ad5-9c64363f80fc None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:35.249425 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119df92c-d272-49d2-8ad5-9c64363f80fc None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:35.249902 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119df92c-d272-49d2-8ad5-9c64363f80fc None admin] PATH_INFO: `/v3/projects/191069caccd04233b012585e29cfa18b/users/0422d6695bf14925bf0f993cb16e64ca/roles/23718d83b0864b16b0071e8ffe0b5861` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:35.271091 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119df92c-d272-49d2-8ad5-9c64363f80fc None admin] RBAC: Authorizing `identity:create_grant(project_id=191069caccd04233b012585e29cfa18b, user_id=0422d6695bf14925bf0f993cb16e64ca, role_id=23718d83b0864b16b0071e8ffe0b5861)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:35.272818 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119df92c-d272-49d2-8ad5-9c64363f80fc None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:35.291983 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 605/1232] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Jul 25 19:12:35 2022] PUT /identity/v3/projects/191069caccd04233b012585e29cfa18b/users/0422d6695bf14925bf0f993cb16e64ca/roles/23718d83b0864b16b0071e8ffe0b5861 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:12:37.653139 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a0cc6-ae23-4810-b0c5-c0d3bf17677f None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:37.653834 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a0cc6-ae23-4810-b0c5-c0d3bf17677f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:37.653834 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a0cc6-ae23-4810-b0c5-c0d3bf17677f None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:37.655338 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 628/1233] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:12:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:12:37.662366 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf5b58-811f-4bfe-8a8e-88acda4a0c48 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:37.662996 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf5b58-811f-4bfe-8a8e-88acda4a0c48 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:37.663174 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf5b58-811f-4bfe-8a8e-88acda4a0c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:37.695093 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1daf5b58-811f-4bfe-8a8e-88acda4a0c48 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:12:37.697239 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1daf5b58-811f-4bfe-8a8e-88acda4a0c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:12:37.846554 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 606/1234] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:12:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 19:12:37.872670 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a0cc6-ae23-4810-b0c5-c0d3bf17677f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:37.904620 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:38.051187 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a0cc6-ae23-4810-b0c5-c0d3bf17677f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:38.116222 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e611a849-93c3-4891-9b18-616588c94b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:38.117595 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611a849-93c3-4891-9b18-616588c94b3a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:38.118087 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611a849-93c3-4891-9b18-616588c94b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:38.118584 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611a849-93c3-4891-9b18-616588c94b3a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:38.139232 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e611a849-93c3-4891-9b18-616588c94b3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:12:38.148451 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e611a849-93c3-4891-9b18-616588c94b3a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:12:38.148451 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:12:38.148451 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:12:38.148451 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:12:38.148451 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:12:38.148451 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:12:38.148451 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:12:38.148451 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:12:38.148451 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:12:38.148451 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:12:38.148451 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:12:38.148451 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:12:38.148451 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:12:38.148451 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:12:38.148451 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:12:38.148451 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:12:38.150667 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:12:38.150667 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:12:38.150667 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:12:38.150667 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:12:38.150667 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:12:38.150667 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:12:38.150667 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:12:38.150667 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:12:38.150667 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:12:38.150667 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:12:38.150667 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:12:38.150667 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:12:38.150667 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:12:38.150667 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:12:38.150667 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:12:38.150667 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:12:38.152432 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:12:38.152432 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:12:38.152432 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:12:38.152432 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:12:38.152432 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:12:38.152432 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:12:38.152432 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:12:38.152432 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:12:38.152432 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:12:38.152432 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:12:38.152432 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 19:12:38.152432 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:12:38.152432 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 629/1235] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Jul 25 19:12:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:12:38.159734 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf5b58-811f-4bfe-8a8e-88acda4a0c48 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:38.253254 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:38.341776 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf5b58-811f-4bfe-8a8e-88acda4a0c48 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:38.407175 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a41ddc-4a4a-45e5-bd66-431d1e80a8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:38.408167 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a41ddc-4a4a-45e5-bd66-431d1e80a8d0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:38.408455 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a41ddc-4a4a-45e5-bd66-431d1e80a8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:38.408882 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a41ddc-4a4a-45e5-bd66-431d1e80a8d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:38.409864 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a41ddc-4a4a-45e5-bd66-431d1e80a8d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:12:38.410213 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a41ddc-4a4a-45e5-bd66-431d1e80a8d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:38.412475 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a41ddc-4a4a-45e5-bd66-431d1e80a8d0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:38.423246 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 607/1236] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Jul 25 19:12:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:12:38.433229 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e611a849-93c3-4891-9b18-616588c94b3a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:38.592838 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:38.625446 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e611a849-93c3-4891-9b18-616588c94b3a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:38.686537 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d6980b-ca42-4272-9690-895b750c114a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:38.687811 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6980b-ca42-4272-9690-895b750c114a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:38.688285 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6980b-ca42-4272-9690-895b750c114a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:38.688602 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6980b-ca42-4272-9690-895b750c114a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:38.706399 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d6980b-ca42-4272-9690-895b750c114a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:38.709550 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d6980b-ca42-4272-9690-895b750c114a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:38.739413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-97d6980b-ca42-4272-9690-895b750c114a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:12:38.739413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:12:38.739413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:12:38.739413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:12:38.739413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:12:38.739413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:12:38.739413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:12:38.739413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:12:38.739413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:12:38.739413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:12:38.739413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:12:38.739413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:12:38.739413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:12:38.739413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:12:38.739413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:12:38.743757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:12:38.743757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:12:38.743757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:12:38.743757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:12:38.743757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:12:38.743757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:12:38.743757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:12:38.743757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:12:38.743757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:12:38.743757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:12:38.743757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:12:38.743757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:12:38.743757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:12:38.743757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:12:38.743757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:12:38.743757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:12:38.743757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:12:38.745554 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:12:38.745554 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:12:38.745554 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:12:38.745554 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:12:38.745554 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 630/1237] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:12:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:12:38.749367 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a41ddc-4a4a-45e5-bd66-431d1e80a8d0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:38.916070 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a41ddc-4a4a-45e5-bd66-431d1e80a8d0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:38.927604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:38.978383 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f65b84-f473-42fd-b052-c56d315ed2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:38.979188 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f65b84-f473-42fd-b052-c56d315ed2c3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:38.979644 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f65b84-f473-42fd-b052-c56d315ed2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:38.980112 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f65b84-f473-42fd-b052-c56d315ed2c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:38.981337 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f65b84-f473-42fd-b052-c56d315ed2c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:12:38.981884 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f65b84-f473-42fd-b052-c56d315ed2c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:38.984504 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f65b84-f473-42fd-b052-c56d315ed2c3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:39.001478 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 608/1238] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:12:39.008674 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d6980b-ca42-4272-9690-895b750c114a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:39.150969 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d6980b-ca42-4272-9690-895b750c114a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:39.208408 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a75027-b491-493a-a2c3-7c1723163cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:39.209443 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a75027-b491-493a-a2c3-7c1723163cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:39.209827 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a75027-b491-493a-a2c3-7c1723163cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:39.210307 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a75027-b491-493a-a2c3-7c1723163cb2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:39.214637 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:39.222697 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a75027-b491-493a-a2c3-7c1723163cb2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:39.224363 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a75027-b491-493a-a2c3-7c1723163cb2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:39.251999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-66a75027-b491-493a-a2c3-7c1723163cb2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 19:12:39.251999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:12:39.251999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:12:39.251999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:12:39.251999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:12:39.251999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:12:39.251999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:12:39.251999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:12:39.251999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:12:39.251999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:12:39.251999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:12:39.251999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:12:39.251999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:12:39.251999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:12:39.251999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:12:39.251999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:12:39.254454 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:12:39.254454 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:12:39.254454 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:12:39.254454 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:12:39.254454 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:12:39.254454 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:12:39.254454 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:12:39.254454 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:12:39.254454 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:12:39.254454 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:12:39.254454 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:12:39.254454 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:12:39.254454 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:12:39.254454 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:12:39.254454 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:12:39.254454 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:12:39.254454 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:12:39.254454 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:12:39.256387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:12:39.256387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:12:39.256387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:12:39.256387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:12:39.256387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:12:39.256387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:12:39.256387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:12:39.256387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:12:39.256387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 19:12:39.256387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:12:39.256387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 631/1239] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:12:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:12:39.263134 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f65b84-f473-42fd-b052-c56d315ed2c3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:39.410868 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f65b84-f473-42fd-b052-c56d315ed2c3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:39.475804 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365bf80e-3543-4ae7-8b5b-0a9810bd3f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:39.476883 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365bf80e-3543-4ae7-8b5b-0a9810bd3f65 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:39.477303 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365bf80e-3543-4ae7-8b5b-0a9810bd3f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:39.477762 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365bf80e-3543-4ae7-8b5b-0a9810bd3f65 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:39.479299 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365bf80e-3543-4ae7-8b5b-0a9810bd3f65 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:12:39.479913 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365bf80e-3543-4ae7-8b5b-0a9810bd3f65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:39.482993 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365bf80e-3543-4ae7-8b5b-0a9810bd3f65 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:39.514250 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 609/1240] 10.222.0.18 () {56 vars in 1143 bytes} [Mon Jul 25 19:12:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:12:39.521159 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a75027-b491-493a-a2c3-7c1723163cb2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:39.680536 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a75027-b491-493a-a2c3-7c1723163cb2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:39.734977 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c46c1-7ec2-43eb-a8e6-d194cc6e0d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:39.735719 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c46c1-7ec2-43eb-a8e6-d194cc6e0d2d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:39.735999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c46c1-7ec2-43eb-a8e6-d194cc6e0d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:39.736281 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c46c1-7ec2-43eb-a8e6-d194cc6e0d2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:39.826900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:39.835917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c46c1-7ec2-43eb-a8e6-d194cc6e0d2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:39.837462 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c46c1-7ec2-43eb-a8e6-d194cc6e0d2d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:39.981468 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 632/1241] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:12:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 19:12:39.990252 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365bf80e-3543-4ae7-8b5b-0a9810bd3f65 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:40.035604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:40.181105 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365bf80e-3543-4ae7-8b5b-0a9810bd3f65 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:40.242350 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b31d75-f783-4cef-972f-e094696d4b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:40.243752 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b31d75-f783-4cef-972f-e094696d4b26 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:40.244048 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b31d75-f783-4cef-972f-e094696d4b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:40.244328 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b31d75-f783-4cef-972f-e094696d4b26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:40.260272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b31d75-f783-4cef-972f-e094696d4b26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:40.263877 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b31d75-f783-4cef-972f-e094696d4b26 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:40.298928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-f3b31d75-f783-4cef-972f-e094696d4b26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:12:40.298928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:12:40.298928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:12:40.298928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:12:40.298928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:12:40.298928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:12:40.298928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:12:40.298928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:12:40.298928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:12:40.298928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:12:40.298928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:12:40.298928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:12:40.298928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:12:40.298928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:12:40.298928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:12:40.300374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:12:40.300374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:12:40.300374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:12:40.300374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:12:40.300374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:12:40.300374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:12:40.300374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:12:40.300374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:12:40.300374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:12:40.300374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:12:40.300374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:12:40.300374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:12:40.300374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:12:40.300374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:12:40.300374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:12:40.300374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:12:40.300374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:12:40.301890 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:12:40.301890 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:12:40.301890 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:12:40.301890 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:12:40.301890 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 610/1242] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:12:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:12:40.308361 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c46c1-7ec2-43eb-a8e6-d194cc6e0d2d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:40.458680 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:40.472528 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c46c1-7ec2-43eb-a8e6-d194cc6e0d2d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:40.546939 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f6a60-1def-4bfb-a9e4-5780d17cb935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:40.548451 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f6a60-1def-4bfb-a9e4-5780d17cb935 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:40.548974 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f6a60-1def-4bfb-a9e4-5780d17cb935 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:40.549588 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f6a60-1def-4bfb-a9e4-5780d17cb935 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:40.552212 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f6a60-1def-4bfb-a9e4-5780d17cb935 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:12:40.552877 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f6a60-1def-4bfb-a9e4-5780d17cb935 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:40.556311 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f6a60-1def-4bfb-a9e4-5780d17cb935 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:40.593190 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 633/1243] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:12:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:12:40.601150 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b31d75-f783-4cef-972f-e094696d4b26 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:40.782218 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:40.825402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b31d75-f783-4cef-972f-e094696d4b26 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:40.877046 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb981f3-a2a3-4be1-ac48-90aa28d77315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:40.877811 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb981f3-a2a3-4be1-ac48-90aa28d77315 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:40.878227 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb981f3-a2a3-4be1-ac48-90aa28d77315 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:40.878533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb981f3-a2a3-4be1-ac48-90aa28d77315 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:40.892007 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb981f3-a2a3-4be1-ac48-90aa28d77315 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:40.894715 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb981f3-a2a3-4be1-ac48-90aa28d77315 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:40.916416 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-0eb981f3-a2a3-4be1-ac48-90aa28d77315 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:12:40.916416 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:12:40.916416 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:12:40.916416 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:12:40.916416 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:12:40.916416 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:12:40.916416 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:12:40.916416 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:12:40.916416 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:12:40.916416 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:12:40.916416 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:12:40.916416 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:12:40.916416 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:12:40.916416 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:12:40.916416 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:12:40.918555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:12:40.918555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:12:40.918555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:12:40.918555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:12:40.918555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:12:40.918555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:12:40.918555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:12:40.918555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:12:40.918555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:12:40.918555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:12:40.918555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:12:40.918555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:12:40.918555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:12:40.918555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:12:40.918555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:12:40.918555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:12:40.918555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:12:40.920724 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:12:40.920724 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:12:40.920724 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:12:40.920724 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:12:40.920724 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:12:40.920724 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:12:40.920724 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:12:40.920724 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:12:40.920724 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 611/1244] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:12:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:12:40.927856 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f6a60-1def-4bfb-a9e4-5780d17cb935 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:41.141738 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f6a60-1def-4bfb-a9e4-5780d17cb935 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:41.221329 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef2056d-2962-4a2e-8c3b-805d70780ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:41.222058 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef2056d-2962-4a2e-8c3b-805d70780ccc None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:41.222519 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef2056d-2962-4a2e-8c3b-805d70780ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:41.223251 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef2056d-2962-4a2e-8c3b-805d70780ccc None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:41.224242 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef2056d-2962-4a2e-8c3b-805d70780ccc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:12:41.224832 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef2056d-2962-4a2e-8c3b-805d70780ccc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:41.227212 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef2056d-2962-4a2e-8c3b-805d70780ccc None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:41.262373 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 634/1245] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Jul 25 19:12:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:12:41.264671 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb981f3-a2a3-4be1-ac48-90aa28d77315 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:41.430651 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb981f3-a2a3-4be1-ac48-90aa28d77315 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:41.485887 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216b541-6ee2-4bc7-bba2-df362e0576eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:41.487081 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216b541-6ee2-4bc7-bba2-df362e0576eb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:41.487618 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216b541-6ee2-4bc7-bba2-df362e0576eb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:41.488116 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216b541-6ee2-4bc7-bba2-df362e0576eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:41.490127 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6216b541-6ee2-4bc7-bba2-df362e0576eb None admin] RBAC: Adding query filter params (role.id=23718d83b0864b16b0071e8ffe0b5861, scope.project.id=191069caccd04233b012585e29cfa18b, user.id=0422d6695bf14925bf0f993cb16e64ca) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:12:41.490659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6216b541-6ee2-4bc7-bba2-df362e0576eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:41.493450 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6216b541-6ee2-4bc7-bba2-df362e0576eb None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:41.502351 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 612/1246] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:12:41 2022] GET /identity/v3/role_assignments?user.id=0422d6695bf14925bf0f993cb16e64ca&scope.project.id=191069caccd04233b012585e29cfa18b&role.id=23718d83b0864b16b0071e8ffe0b5861 => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:12:44.190592 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a9c25-7f48-440f-907f-ee33ed99e884 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:44.192184 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a9c25-7f48-440f-907f-ee33ed99e884 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:44.192336 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a9c25-7f48-440f-907f-ee33ed99e884 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:44.195577 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 635/1247] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:12:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:12:44.209387 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846f677-a116-4217-be0d-f213a1d0f98c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:44.210096 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846f677-a116-4217-be0d-f213a1d0f98c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:44.210096 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846f677-a116-4217-be0d-f213a1d0f98c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:44.218282 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:44.240133 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b846f677-a116-4217-be0d-f213a1d0f98c None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:12:44.241314 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b846f677-a116-4217-be0d-f213a1d0f98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:12:44.357475 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 613/1248] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:12:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 19:12:44.388142 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a9c25-7f48-440f-907f-ee33ed99e884 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:44.413477 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:44.549008 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a9c25-7f48-440f-907f-ee33ed99e884 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:44.618817 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5063ed6f-2207-4c2a-9095-c8d01df6cb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:44.620280 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063ed6f-2207-4c2a-9095-c8d01df6cb45 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:44.620654 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063ed6f-2207-4c2a-9095-c8d01df6cb45 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:44.621388 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063ed6f-2207-4c2a-9095-c8d01df6cb45 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:44.646817 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5063ed6f-2207-4c2a-9095-c8d01df6cb45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:12:44.658154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-5063ed6f-2207-4c2a-9095-c8d01df6cb45 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:12:44.658154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:12:44.658154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:12:44.658154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:12:44.658154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:12:44.658154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:12:44.658154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:12:44.658154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:12:44.658154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:12:44.658154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:12:44.658154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:12:44.658154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:12:44.658154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:12:44.658154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:12:44.658154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:12:44.658154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:12:44.660589 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:12:44.660589 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:12:44.660589 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:12:44.660589 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:12:44.660589 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:12:44.660589 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:12:44.660589 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:12:44.660589 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:12:44.660589 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:12:44.660589 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:12:44.660589 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:12:44.660589 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:12:44.660589 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:12:44.660589 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:12:44.660589 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:12:44.660589 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:12:44.663633 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:12:44.663633 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:12:44.663633 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:12:44.663633 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:12:44.663633 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:12:44.663633 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:12:44.663633 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:12:44.663633 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:12:44.663633 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:12:44.663633 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:12:44.663633 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:12:44.663633 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:12:44.664947 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 636/1249] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Jul 25 19:12:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:12:44.672197 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846f677-a116-4217-be0d-f213a1d0f98c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:44.822842 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846f677-a116-4217-be0d-f213a1d0f98c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:44.862061 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:44.869289 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff0076-281f-4c42-9c23-2e0490277fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:44.869994 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff0076-281f-4c42-9c23-2e0490277fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:44.870303 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff0076-281f-4c42-9c23-2e0490277fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:44.870658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff0076-281f-4c42-9c23-2e0490277fa0 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:44.871450 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff0076-281f-4c42-9c23-2e0490277fa0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:12:44.871802 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff0076-281f-4c42-9c23-2e0490277fa0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:44.873318 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff0076-281f-4c42-9c23-2e0490277fa0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:44.884177 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 614/1250] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:12:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:12:44.896284 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5063ed6f-2207-4c2a-9095-c8d01df6cb45 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:45.062676 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:45.105960 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5063ed6f-2207-4c2a-9095-c8d01df6cb45 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:45.183737 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9fdc6-b9e0-45c3-ba92-b0ebf6e732da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:45.185219 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9fdc6-b9e0-45c3-ba92-b0ebf6e732da None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:45.185809 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9fdc6-b9e0-45c3-ba92-b0ebf6e732da None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:45.186486 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9fdc6-b9e0-45c3-ba92-b0ebf6e732da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:45.211764 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb9fdc6-b9e0-45c3-ba92-b0ebf6e732da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:45.214727 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb9fdc6-b9e0-45c3-ba92-b0ebf6e732da None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:45.244200 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-4fb9fdc6-b9e0-45c3-ba92-b0ebf6e732da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:12:45.244200 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:12:45.244200 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:12:45.244200 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:12:45.244200 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:12:45.244200 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:12:45.244200 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:12:45.244200 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:12:45.244200 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:12:45.244200 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:12:45.244200 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:12:45.244200 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:12:45.244200 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:12:45.244200 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:12:45.244200 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:12:45.246425 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:12:45.246425 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:12:45.246425 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:12:45.246425 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:12:45.246425 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:12:45.246425 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:12:45.246425 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:12:45.246425 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:12:45.246425 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:12:45.246425 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:12:45.246425 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:12:45.246425 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:12:45.246425 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:12:45.246425 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:12:45.246425 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:12:45.246425 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:12:45.246425 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:12:45.248545 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:12:45.248545 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:12:45.248545 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:12:45.248545 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:12:45.248545 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 637/1251] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:12:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:12:45.256918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff0076-281f-4c42-9c23-2e0490277fa0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:45.482397 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff0076-281f-4c42-9c23-2e0490277fa0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:45.560712 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e7633-4530-4191-b827-9925b0505605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:45.562385 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e7633-4530-4191-b827-9925b0505605 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:45.562891 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e7633-4530-4191-b827-9925b0505605 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:45.563583 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e7633-4530-4191-b827-9925b0505605 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:45.565664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719e7633-4530-4191-b827-9925b0505605 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:12:45.566809 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719e7633-4530-4191-b827-9925b0505605 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:45.570130 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719e7633-4530-4191-b827-9925b0505605 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:45.603660 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 615/1252] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:12:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:12:45.613424 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9fdc6-b9e0-45c3-ba92-b0ebf6e732da None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:45.817877 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9fdc6-b9e0-45c3-ba92-b0ebf6e732da None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:45.853334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:45.863009 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92d367d-5a9f-4a56-9258-d2d848986dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:45.864423 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d367d-5a9f-4a56-9258-d2d848986dde None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:45.864728 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d367d-5a9f-4a56-9258-d2d848986dde None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:45.865149 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d367d-5a9f-4a56-9258-d2d848986dde None admin] PATH_INFO: `/v3/users/placement` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:45.874374 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92d367d-5a9f-4a56-9258-d2d848986dde None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:45.876053 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92d367d-5a9f-4a56-9258-d2d848986dde None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:45.893839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-c92d367d-5a9f-4a56-9258-d2d848986dde None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 19:12:45.893839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:12:45.893839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:12:45.893839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:12:45.893839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:12:45.893839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:12:45.893839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:12:45.893839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:12:45.893839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:12:45.893839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:12:45.893839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:12:45.893839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:12:45.893839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:12:45.893839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:12:45.893839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:12:45.893839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:12:45.896561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:12:45.896561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:12:45.896561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:12:45.896561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:12:45.896561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:12:45.896561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:12:45.896561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:12:45.896561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:12:45.896561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:12:45.896561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:12:45.896561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:12:45.896561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:12:45.896561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:12:45.896561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:12:45.896561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:12:45.896561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:12:45.896561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:12:45.896561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:12:45.898612 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:12:45.898612 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:12:45.898612 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:12:45.898612 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:12:45.898612 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:12:45.898612 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:12:45.898612 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:12:45.898612 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:12:45.898612 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 19:12:45.898612 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:12:45.898612 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 638/1253] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:12:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:12:45.905128 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e7633-4530-4191-b827-9925b0505605 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:46.005339 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:46.103783 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e7633-4530-4191-b827-9925b0505605 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:46.167038 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b63129-9ef0-48eb-bbe6-223afbcc7edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:46.167680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b63129-9ef0-48eb-bbe6-223afbcc7edd None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:46.168193 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b63129-9ef0-48eb-bbe6-223afbcc7edd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:46.168684 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b63129-9ef0-48eb-bbe6-223afbcc7edd None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:46.170369 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b63129-9ef0-48eb-bbe6-223afbcc7edd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:12:46.172247 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b63129-9ef0-48eb-bbe6-223afbcc7edd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:46.173785 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b63129-9ef0-48eb-bbe6-223afbcc7edd None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:46.203588 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 616/1254] 10.222.0.18 () {56 vars in 1143 bytes} [Mon Jul 25 19:12:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:12:46.212334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92d367d-5a9f-4a56-9258-d2d848986dde None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:46.369599 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92d367d-5a9f-4a56-9258-d2d848986dde None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:46.433658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ea779-6dc9-4bc2-83d9-83146027aa73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:46.434784 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ea779-6dc9-4bc2-83d9-83146027aa73 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:46.435290 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ea779-6dc9-4bc2-83d9-83146027aa73 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:46.435828 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ea779-6dc9-4bc2-83d9-83146027aa73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:46.536241 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:46.554454 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100ea779-6dc9-4bc2-83d9-83146027aa73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:46.556817 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100ea779-6dc9-4bc2-83d9-83146027aa73 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:46.748543 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 639/1255] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:12:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 19:12:46.758176 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b63129-9ef0-48eb-bbe6-223afbcc7edd None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:46.966459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b63129-9ef0-48eb-bbe6-223afbcc7edd None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:46.987754 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:47.032203 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a26ba-1824-4ddd-ac00-98351195f618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:47.033220 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a26ba-1824-4ddd-ac00-98351195f618 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:47.033599 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a26ba-1824-4ddd-ac00-98351195f618 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:47.034083 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a26ba-1824-4ddd-ac00-98351195f618 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:47.049625 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17a26ba-1824-4ddd-ac00-98351195f618 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:47.052678 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17a26ba-1824-4ddd-ac00-98351195f618 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:47.077049 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-d17a26ba-1824-4ddd-ac00-98351195f618 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:12:47.077049 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:12:47.077049 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:12:47.077049 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:12:47.077049 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:12:47.077049 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:12:47.077049 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:12:47.077049 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:12:47.077049 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:12:47.077049 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:12:47.077049 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:12:47.077049 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:12:47.077049 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:12:47.077049 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:12:47.077049 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:12:47.079135 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:12:47.079135 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:12:47.079135 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:12:47.079135 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:12:47.079135 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:12:47.079135 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:12:47.079135 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:12:47.079135 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:12:47.079135 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:12:47.079135 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:12:47.079135 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:12:47.079135 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:12:47.079135 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:12:47.079135 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:12:47.079135 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:12:47.079135 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:12:47.079135 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:12:47.081073 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:12:47.081073 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:12:47.081073 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:12:47.081073 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:12:47.081073 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 617/1256] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:12:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:12:47.089762 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ea779-6dc9-4bc2-83d9-83146027aa73 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:47.268870 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:47.293546 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ea779-6dc9-4bc2-83d9-83146027aa73 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:47.367006 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43cdd2-e01a-4703-8946-a2bbb02200b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:47.367830 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43cdd2-e01a-4703-8946-a2bbb02200b7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:47.368117 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43cdd2-e01a-4703-8946-a2bbb02200b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:47.368395 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43cdd2-e01a-4703-8946-a2bbb02200b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:47.369160 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43cdd2-e01a-4703-8946-a2bbb02200b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:12:47.369468 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43cdd2-e01a-4703-8946-a2bbb02200b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:47.370896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43cdd2-e01a-4703-8946-a2bbb02200b7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:47.389480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 640/1257] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:12:47.397987 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a26ba-1824-4ddd-ac00-98351195f618 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:47.556959 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a26ba-1824-4ddd-ac00-98351195f618 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:47.619636 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfbfb7d-c0db-4c64-a98b-cd5996299194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:47.619636 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbfb7d-c0db-4c64-a98b-cd5996299194 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:47.619636 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbfb7d-c0db-4c64-a98b-cd5996299194 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:47.620799 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbfb7d-c0db-4c64-a98b-cd5996299194 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:47.635295 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfbfb7d-c0db-4c64-a98b-cd5996299194 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:47.637129 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfbfb7d-c0db-4c64-a98b-cd5996299194 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:47.657568 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-4bfbfb7d-c0db-4c64-a98b-cd5996299194 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:12:47.657568 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:12:47.657568 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:12:47.657568 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:12:47.657568 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:12:47.657568 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:12:47.657568 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:12:47.657568 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:12:47.657568 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:12:47.657568 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:12:47.657568 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:12:47.657568 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:12:47.657568 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:12:47.657568 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:12:47.657568 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:12:47.661533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:12:47.661533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:12:47.661533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:12:47.661533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:12:47.661533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:12:47.661533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:12:47.661533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:12:47.661533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:12:47.661533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:12:47.661533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:12:47.661533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:12:47.661533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:12:47.661533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:12:47.661533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:12:47.661533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:12:47.661533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:12:47.661533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:12:47.663631 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:12:47.663631 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:12:47.663631 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:12:47.663631 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:12:47.663631 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:12:47.663631 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:12:47.663631 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:12:47.663631 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:12:47.663631 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 618/1258] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:12:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:12:47.668890 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43cdd2-e01a-4703-8946-a2bbb02200b7 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:47.832695 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43cdd2-e01a-4703-8946-a2bbb02200b7 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:47.880399 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5694ad1-385b-44aa-9b90-90678502f21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:47.881424 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5694ad1-385b-44aa-9b90-90678502f21e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:47.881604 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5694ad1-385b-44aa-9b90-90678502f21e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:47.882091 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5694ad1-385b-44aa-9b90-90678502f21e None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:47.883370 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5694ad1-385b-44aa-9b90-90678502f21e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:12:47.883824 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5694ad1-385b-44aa-9b90-90678502f21e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:47.885974 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5694ad1-385b-44aa-9b90-90678502f21e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:47.924938 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 641/1259] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Jul 25 19:12:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:12:47.935022 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfbfb7d-c0db-4c64-a98b-cd5996299194 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:47.971318 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:48.099899 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfbfb7d-c0db-4c64-a98b-cd5996299194 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:48.158498 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a34bb-213f-4624-a887-5fc7a452848d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:48.159715 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a34bb-213f-4624-a887-5fc7a452848d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:48.160310 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a34bb-213f-4624-a887-5fc7a452848d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:48.160824 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a34bb-213f-4624-a887-5fc7a452848d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:48.162797 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50a34bb-213f-4624-a887-5fc7a452848d None admin] RBAC: Adding query filter params (role.id=ed7fec18968241efa76a0ee769d96da9, scope.project.id=191069caccd04233b012585e29cfa18b, user.id=0422d6695bf14925bf0f993cb16e64ca) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:12:48.163415 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50a34bb-213f-4624-a887-5fc7a452848d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:48.166362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50a34bb-213f-4624-a887-5fc7a452848d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:48.191556 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 619/1260] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:12:47 2022] GET /identity/v3/role_assignments?user.id=0422d6695bf14925bf0f993cb16e64ca&scope.project.id=191069caccd04233b012585e29cfa18b&role.id=ed7fec18968241efa76a0ee769d96da9 => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:12:50.842484 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28495baa-7177-4580-a4b6-5c2d964c767f None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:50.843328 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28495baa-7177-4580-a4b6-5c2d964c767f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:50.843570 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28495baa-7177-4580-a4b6-5c2d964c767f None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:50.845068 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 642/1261] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:12:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:12:50.851259 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f6b09-1a77-4dce-a57f-b164ffc419b9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:50.852029 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f6b09-1a77-4dce-a57f-b164ffc419b9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:50.852141 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f6b09-1a77-4dce-a57f-b164ffc419b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:50.873566 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a34f6b09-1a77-4dce-a57f-b164ffc419b9 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:12:50.875064 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a34f6b09-1a77-4dce-a57f-b164ffc419b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:12:50.955786 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 620/1262] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:12:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:12:50.980742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28495baa-7177-4580-a4b6-5c2d964c767f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:50.993265 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:51.101142 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28495baa-7177-4580-a4b6-5c2d964c767f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:51.143674 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a285e6e-edde-45d4-9ad7-63c218ed2d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:51.145097 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a285e6e-edde-45d4-9ad7-63c218ed2d77 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:51.145377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a285e6e-edde-45d4-9ad7-63c218ed2d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:51.145667 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a285e6e-edde-45d4-9ad7-63c218ed2d77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:51.161826 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a285e6e-edde-45d4-9ad7-63c218ed2d77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:12:51.169922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-6a285e6e-edde-45d4-9ad7-63c218ed2d77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:12:51.169922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:12:51.169922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:12:51.169922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:12:51.169922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:12:51.169922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:12:51.169922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:12:51.169922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:12:51.169922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:12:51.169922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:12:51.169922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:12:51.169922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:12:51.169922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:12:51.169922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:12:51.169922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:12:51.169922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:12:51.172130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:12:51.172130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:12:51.172130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:12:51.172130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:12:51.172130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:12:51.172130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:12:51.172130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:12:51.172130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:12:51.172130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:12:51.172130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:12:51.172130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:12:51.172130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:12:51.172130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:12:51.172130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:12:51.172130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:12:51.172130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:12:51.174989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:12:51.174989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:12:51.174989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:12:51.174989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:12:51.174989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:12:51.174989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:12:51.174989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:12:51.174989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:12:51.174989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:12:51.174989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:12:51.174989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:12:51.174989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:12:51.174989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 643/1263] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Jul 25 19:12:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:12:51.179024 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f6b09-1a77-4dce-a57f-b164ffc419b9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:51.207498 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:51.331994 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f6b09-1a77-4dce-a57f-b164ffc419b9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:51.378412 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74cc34-43f0-4b45-b5fe-ddb9acd57c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:51.379873 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74cc34-43f0-4b45-b5fe-ddb9acd57c68 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:51.380249 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74cc34-43f0-4b45-b5fe-ddb9acd57c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:51.380621 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74cc34-43f0-4b45-b5fe-ddb9acd57c68 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:51.381553 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e74cc34-43f0-4b45-b5fe-ddb9acd57c68 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:12:51.381969 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e74cc34-43f0-4b45-b5fe-ddb9acd57c68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:51.383566 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e74cc34-43f0-4b45-b5fe-ddb9acd57c68 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:51.396756 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 621/1264] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:12:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:12:51.407048 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a285e6e-edde-45d4-9ad7-63c218ed2d77 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:51.513289 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:51.587288 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a285e6e-edde-45d4-9ad7-63c218ed2d77 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:51.666731 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19c7da4-b597-4efa-94c1-a6faf4dddba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:51.668067 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19c7da4-b597-4efa-94c1-a6faf4dddba8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:51.668673 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19c7da4-b597-4efa-94c1-a6faf4dddba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:51.669590 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19c7da4-b597-4efa-94c1-a6faf4dddba8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:51.693553 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19c7da4-b597-4efa-94c1-a6faf4dddba8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:51.695857 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19c7da4-b597-4efa-94c1-a6faf4dddba8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:51.723641 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-a19c7da4-b597-4efa-94c1-a6faf4dddba8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:12:51.723641 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:12:51.723641 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:12:51.723641 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:12:51.723641 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:12:51.723641 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:12:51.723641 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:12:51.723641 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:12:51.723641 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:12:51.723641 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:12:51.723641 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:12:51.723641 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:12:51.723641 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:12:51.723641 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:12:51.723641 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:12:51.725683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:12:51.725683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:12:51.725683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:12:51.725683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:12:51.725683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:12:51.725683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:12:51.725683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:12:51.725683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:12:51.725683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:12:51.725683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:12:51.725683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:12:51.725683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:12:51.725683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:12:51.725683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:12:51.725683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:12:51.725683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:12:51.725683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:12:51.727790 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:12:51.727790 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:12:51.727790 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:12:51.727790 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:12:51.727790 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 644/1265] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:12:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:12:51.732135 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74cc34-43f0-4b45-b5fe-ddb9acd57c68 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:51.846360 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:51.873676 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74cc34-43f0-4b45-b5fe-ddb9acd57c68 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:51.921376 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1913c-66ef-43d5-9c94-612459e62e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:51.921980 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1913c-66ef-43d5-9c94-612459e62e66 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:51.922267 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1913c-66ef-43d5-9c94-612459e62e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:51.922547 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1913c-66ef-43d5-9c94-612459e62e66 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:51.923438 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df1913c-66ef-43d5-9c94-612459e62e66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:12:51.923783 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df1913c-66ef-43d5-9c94-612459e62e66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:51.925843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df1913c-66ef-43d5-9c94-612459e62e66 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:51.940605 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 622/1266] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:12:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:12:51.948027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19c7da4-b597-4efa-94c1-a6faf4dddba8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:52.089385 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19c7da4-b597-4efa-94c1-a6faf4dddba8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:52.116559 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:52.137014 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e37a17-86b0-4482-9761-a3cb01ae6de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:52.137938 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e37a17-86b0-4482-9761-a3cb01ae6de4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:52.138326 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e37a17-86b0-4482-9761-a3cb01ae6de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:52.138680 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e37a17-86b0-4482-9761-a3cb01ae6de4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:52.151060 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e37a17-86b0-4482-9761-a3cb01ae6de4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:52.153244 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e37a17-86b0-4482-9761-a3cb01ae6de4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:52.173360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-11e37a17-86b0-4482-9761-a3cb01ae6de4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 19:12:52.173360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:12:52.173360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:12:52.173360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:12:52.173360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:12:52.173360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:12:52.173360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:12:52.173360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:12:52.173360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:12:52.173360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:12:52.173360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:12:52.173360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:12:52.173360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:12:52.173360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:12:52.173360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:12:52.173360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:12:52.174563 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:12:52.174563 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:12:52.174563 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:12:52.174563 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:12:52.174563 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:12:52.174563 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:12:52.174563 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:12:52.174563 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:12:52.174563 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:12:52.174563 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:12:52.174563 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:12:52.174563 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:12:52.174563 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:12:52.174563 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:12:52.174563 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:12:52.174563 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:12:52.174563 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:12:52.174563 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:12:52.175877 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:12:52.175877 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:12:52.175877 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:12:52.175877 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:12:52.175877 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:12:52.175877 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:12:52.175877 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:12:52.175877 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:12:52.175877 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 19:12:52.175877 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:12:52.175877 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 645/1267] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:12:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:12:52.185674 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1913c-66ef-43d5-9c94-612459e62e66 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:52.343753 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1913c-66ef-43d5-9c94-612459e62e66 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:52.399820 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5312c2e-7ba2-4886-a716-f12521dfcdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:52.400892 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5312c2e-7ba2-4886-a716-f12521dfcdae None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:52.401316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5312c2e-7ba2-4886-a716-f12521dfcdae None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:52.401737 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5312c2e-7ba2-4886-a716-f12521dfcdae None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:52.403158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5312c2e-7ba2-4886-a716-f12521dfcdae None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:12:52.403663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5312c2e-7ba2-4886-a716-f12521dfcdae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:52.406007 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5312c2e-7ba2-4886-a716-f12521dfcdae None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:52.421494 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:52.434974 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 623/1268] 10.222.0.18 () {56 vars in 1143 bytes} [Mon Jul 25 19:12:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:12:52.442129 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e37a17-86b0-4482-9761-a3cb01ae6de4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:52.618976 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e37a17-86b0-4482-9761-a3cb01ae6de4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:52.690010 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3554c54-c4f2-4028-a393-055076807db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:52.692305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3554c54-c4f2-4028-a393-055076807db5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:52.692802 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3554c54-c4f2-4028-a393-055076807db5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:52.693302 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3554c54-c4f2-4028-a393-055076807db5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:52.714487 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3554c54-c4f2-4028-a393-055076807db5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:52.717602 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3554c54-c4f2-4028-a393-055076807db5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:52.757269 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b3554c54-c4f2-4028-a393-055076807db5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:12:52.757269 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:12:52.757269 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:12:52.757269 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:12:52.757269 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:12:52.757269 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:12:52.757269 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:12:52.757269 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:12:52.757269 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:12:52.757269 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:12:52.757269 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:12:52.757269 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:12:52.757269 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:12:52.757269 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:12:52.757269 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:12:52.758925 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:12:52.758925 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:12:52.758925 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:12:52.758925 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:12:52.758925 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:12:52.758925 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:12:52.758925 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:12:52.758925 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:12:52.758925 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:12:52.758925 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:12:52.758925 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:12:52.758925 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:12:52.758925 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:12:52.758925 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:12:52.758925 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:12:52.758925 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:12:52.758925 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:12:52.761116 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:12:52.761116 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:12:52.761116 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:12:52.761116 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:12:52.761116 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 646/1269] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:12:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:12:52.766687 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5312c2e-7ba2-4886-a716-f12521dfcdae None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:52.949751 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5312c2e-7ba2-4886-a716-f12521dfcdae None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:53.017865 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c0eb9-a5c4-401b-a7dd-d95db3b5e233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:53.018765 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c0eb9-a5c4-401b-a7dd-d95db3b5e233 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:53.019775 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c0eb9-a5c4-401b-a7dd-d95db3b5e233 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:53.020125 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c0eb9-a5c4-401b-a7dd-d95db3b5e233 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:53.021020 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7c0eb9-a5c4-401b-a7dd-d95db3b5e233 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:12:53.021406 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7c0eb9-a5c4-401b-a7dd-d95db3b5e233 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:53.022967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7c0eb9-a5c4-401b-a7dd-d95db3b5e233 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:53.037777 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 624/1270] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:12:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:12:53.053230 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3554c54-c4f2-4028-a393-055076807db5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:53.106272 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:53.248662 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3554c54-c4f2-4028-a393-055076807db5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:53.313936 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e736a-c9f8-4f49-891b-e2a3a413fe02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:53.315248 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e736a-c9f8-4f49-891b-e2a3a413fe02 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:53.315793 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e736a-c9f8-4f49-891b-e2a3a413fe02 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:53.316737 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e736a-c9f8-4f49-891b-e2a3a413fe02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:53.330527 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62e736a-c9f8-4f49-891b-e2a3a413fe02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:53.333872 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62e736a-c9f8-4f49-891b-e2a3a413fe02 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:53.362150 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d62e736a-c9f8-4f49-891b-e2a3a413fe02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:12:53.362150 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:12:53.362150 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:12:53.362150 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:12:53.362150 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:12:53.362150 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:12:53.362150 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:12:53.362150 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:12:53.362150 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:12:53.362150 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:12:53.362150 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:12:53.362150 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:12:53.362150 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:12:53.362150 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:12:53.362150 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:12:53.363216 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:12:53.363216 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:12:53.363216 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:12:53.363216 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:12:53.363216 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:12:53.363216 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:12:53.363216 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:12:53.363216 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:12:53.363216 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:12:53.363216 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:12:53.363216 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:12:53.363216 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:12:53.363216 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:12:53.363216 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:12:53.363216 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:12:53.363216 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:12:53.363216 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:12:53.364185 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:12:53.364185 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:12:53.364185 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:12:53.364185 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:12:53.364185 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:12:53.364185 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:12:53.364185 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:12:53.364185 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:12:53.364775 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 647/1271] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:12:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:12:53.374043 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c0eb9-a5c4-401b-a7dd-d95db3b5e233 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:53.487453 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:53.554226 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c0eb9-a5c4-401b-a7dd-d95db3b5e233 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:53.617881 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2162332-bb07-4411-82b5-36645ac47460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:53.619158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2162332-bb07-4411-82b5-36645ac47460 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:53.619671 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2162332-bb07-4411-82b5-36645ac47460 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:53.620446 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2162332-bb07-4411-82b5-36645ac47460 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:53.622052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2162332-bb07-4411-82b5-36645ac47460 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:12:53.622699 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2162332-bb07-4411-82b5-36645ac47460 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:53.625361 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2162332-bb07-4411-82b5-36645ac47460 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:53.641511 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 625/1272] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Jul 25 19:12:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:12:53.648633 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e736a-c9f8-4f49-891b-e2a3a413fe02 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:53.806591 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:53.826855 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e736a-c9f8-4f49-891b-e2a3a413fe02 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:53.883456 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77b74f-8026-4924-b30b-2cdd28c2f115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:53.884541 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77b74f-8026-4924-b30b-2cdd28c2f115 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:53.884918 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77b74f-8026-4924-b30b-2cdd28c2f115 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:53.885294 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77b74f-8026-4924-b30b-2cdd28c2f115 None admin] PATH_INFO: `/v3/projects/191069caccd04233b012585e29cfa18b/users/0422d6695bf14925bf0f993cb16e64ca/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:53.888211 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77b74f-8026-4924-b30b-2cdd28c2f115 None admin] RBAC: Authorizing `identity:create_grant(project_id=191069caccd04233b012585e29cfa18b, user_id=0422d6695bf14925bf0f993cb16e64ca, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:53.890139 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77b74f-8026-4924-b30b-2cdd28c2f115 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:53.911486 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 648/1273] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Jul 25 19:12:53 2022] PUT /identity/v3/projects/191069caccd04233b012585e29cfa18b/users/0422d6695bf14925bf0f993cb16e64ca/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:12:56.229422 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b91a7f-4868-401c-8356-478ae186ec32 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:56.230042 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b91a7f-4868-401c-8356-478ae186ec32 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:56.230042 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b91a7f-4868-401c-8356-478ae186ec32 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:56.231432 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 626/1274] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:12:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:12:56.237914 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90340c39-03e0-4495-b6f2-f85f91fed1f6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:56.238990 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90340c39-03e0-4495-b6f2-f85f91fed1f6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:56.239248 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90340c39-03e0-4495-b6f2-f85f91fed1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:56.260139 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-90340c39-03e0-4495-b6f2-f85f91fed1f6 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:12:56.261463 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-90340c39-03e0-4495-b6f2-f85f91fed1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:12:56.348325 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 649/1275] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:12:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:12:56.390557 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b91a7f-4868-401c-8356-478ae186ec32 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:56.561257 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b91a7f-4868-401c-8356-478ae186ec32 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:56.592360 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:56.624626 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06804cc-56a2-4773-aadb-877203452c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:56.625606 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06804cc-56a2-4773-aadb-877203452c7a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:56.626067 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06804cc-56a2-4773-aadb-877203452c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:56.626441 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06804cc-56a2-4773-aadb-877203452c7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:56.655912 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06804cc-56a2-4773-aadb-877203452c7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 19:12:56.665843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-c06804cc-56a2-4773-aadb-877203452c7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:12:56.665843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:12:56.665843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:12:56.665843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:12:56.665843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:12:56.665843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:12:56.665843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:12:56.665843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:12:56.665843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:12:56.665843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:12:56.665843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:12:56.665843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:12:56.665843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 19:12:56.665843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 19:12:56.665843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 19:12:56.665843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 19:12:56.667928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 19:12:56.667928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 19:12:56.667928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 19:12:56.667928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 19:12:56.667928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:12:56.667928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:12:56.667928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:12:56.667928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:12:56.667928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:12:56.667928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:12:56.667928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:12:56.667928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:12:56.667928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:12:56.667928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:12:56.667928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:12:56.667928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:12:56.669386 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:12:56.669386 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:12:56.669386 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:12:56.669386 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:12:56.669386 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 19:12:56.669386 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 19:12:56.669386 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 19:12:56.669386 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 19:12:56.669386 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 19:12:56.669386 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 19:12:56.669386 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 19:12:56.669386 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:12:56.669386 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 627/1276] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Jul 25 19:12:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:12:56.679445 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90340c39-03e0-4495-b6f2-f85f91fed1f6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:56.835301 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:56.867089 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90340c39-03e0-4495-b6f2-f85f91fed1f6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:56.920297 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35031140-439b-4f79-910c-dfba587fad15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:56.921202 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35031140-439b-4f79-910c-dfba587fad15 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:56.921648 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35031140-439b-4f79-910c-dfba587fad15 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:56.922056 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35031140-439b-4f79-910c-dfba587fad15 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:56.923565 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35031140-439b-4f79-910c-dfba587fad15 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:12:56.924043 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35031140-439b-4f79-910c-dfba587fad15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:56.926022 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35031140-439b-4f79-910c-dfba587fad15 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:56.936783 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 650/1277] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Jul 25 19:12:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:12:56.944211 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06804cc-56a2-4773-aadb-877203452c7a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:57.077101 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06804cc-56a2-4773-aadb-877203452c7a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:57.122649 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e158213-3099-4b2e-8898-ecd109c7d409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:57.123445 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e158213-3099-4b2e-8898-ecd109c7d409 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:57.123759 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e158213-3099-4b2e-8898-ecd109c7d409 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:57.124035 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e158213-3099-4b2e-8898-ecd109c7d409 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:57.138053 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e158213-3099-4b2e-8898-ecd109c7d409 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:57.139855 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e158213-3099-4b2e-8898-ecd109c7d409 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:57.162435 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-3e158213-3099-4b2e-8898-ecd109c7d409 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:12:57.162435 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:12:57.162435 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:12:57.162435 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:12:57.162435 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:12:57.162435 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:12:57.162435 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:12:57.162435 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:12:57.162435 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:12:57.162435 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:12:57.162435 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:12:57.162435 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:12:57.162435 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:12:57.162435 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:12:57.162435 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:12:57.164316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:12:57.164316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:12:57.164316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:12:57.164316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:12:57.164316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:12:57.164316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:12:57.164316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:12:57.164316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:12:57.164316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:12:57.164316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:12:57.164316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:12:57.164316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:12:57.164316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:12:57.164316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:12:57.164316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:12:57.164316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:12:57.164316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:12:57.166068 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:12:57.166068 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:12:57.166068 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:12:57.166068 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:12:57.166068 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 628/1278] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:12:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:12:57.172765 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35031140-439b-4f79-910c-dfba587fad15 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:57.302129 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35031140-439b-4f79-910c-dfba587fad15 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:57.346497 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c6464-abdb-4b9b-82a1-553570b480ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:57.347878 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c6464-abdb-4b9b-82a1-553570b480ca None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:57.348295 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c6464-abdb-4b9b-82a1-553570b480ca None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:57.348656 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c6464-abdb-4b9b-82a1-553570b480ca None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:57.349474 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c6464-abdb-4b9b-82a1-553570b480ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:12:57.349858 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c6464-abdb-4b9b-82a1-553570b480ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:57.351376 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c6464-abdb-4b9b-82a1-553570b480ca None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:57.366042 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 651/1279] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:12:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:12:57.373924 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e158213-3099-4b2e-8898-ecd109c7d409 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:57.419445 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:57.512263 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e158213-3099-4b2e-8898-ecd109c7d409 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:57.554819 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc89ba-c52e-4221-bf10-4bc826cab984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:57.555748 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc89ba-c52e-4221-bf10-4bc826cab984 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:57.556160 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc89ba-c52e-4221-bf10-4bc826cab984 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:57.556407 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc89ba-c52e-4221-bf10-4bc826cab984 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:57.565615 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc89ba-c52e-4221-bf10-4bc826cab984 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:57.568155 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc89ba-c52e-4221-bf10-4bc826cab984 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:57.591751 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-73fc89ba-c52e-4221-bf10-4bc826cab984 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 19:12:57.591751 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:12:57.591751 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:12:57.591751 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:12:57.591751 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:12:57.591751 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:12:57.591751 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:12:57.591751 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:12:57.591751 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:12:57.591751 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:12:57.591751 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:12:57.591751 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:12:57.591751 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 19:12:57.591751 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 19:12:57.591751 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 19:12:57.591751 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 19:12:57.594321 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:12:57.594321 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:12:57.594321 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 19:12:57.594321 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:12:57.594321 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 19:12:57.594321 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 19:12:57.594321 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:12:57.594321 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:12:57.594321 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:12:57.594321 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:12:57.594321 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:12:57.594321 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:12:57.594321 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:12:57.594321 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:12:57.594321 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:12:57.594321 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:12:57.594321 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:12:57.594321 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:12:57.596650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:12:57.596650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:12:57.596650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 19:12:57.596650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 19:12:57.596650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 19:12:57.596650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 19:12:57.596650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 19:12:57.596650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 19:12:57.596650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 19:12:57.596650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:12:57.596650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 629/1280] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:12:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:12:57.600572 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c6464-abdb-4b9b-82a1-553570b480ca None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:57.605105 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:57.778588 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c6464-abdb-4b9b-82a1-553570b480ca None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:57.824031 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d043b6a-5cf1-4279-a1bb-192d66711582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:57.824720 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d043b6a-5cf1-4279-a1bb-192d66711582 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:57.825005 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d043b6a-5cf1-4279-a1bb-192d66711582 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:57.825283 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d043b6a-5cf1-4279-a1bb-192d66711582 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:57.826158 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d043b6a-5cf1-4279-a1bb-192d66711582 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:12:57.826467 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d043b6a-5cf1-4279-a1bb-192d66711582 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:57.827997 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d043b6a-5cf1-4279-a1bb-192d66711582 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:57.850192 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 652/1281] 10.222.0.18 () {56 vars in 1143 bytes} [Mon Jul 25 19:12:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:12:57.856287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc89ba-c52e-4221-bf10-4bc826cab984 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:57.991069 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:58.010055 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc89ba-c52e-4221-bf10-4bc826cab984 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:58.079441 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d67f28-63db-4ef6-bc72-70f2ac093999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:58.080329 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d67f28-63db-4ef6-bc72-70f2ac093999 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:58.080620 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d67f28-63db-4ef6-bc72-70f2ac093999 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:58.080901 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d67f28-63db-4ef6-bc72-70f2ac093999 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:58.193801 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d67f28-63db-4ef6-bc72-70f2ac093999 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:58.195326 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d67f28-63db-4ef6-bc72-70f2ac093999 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:58.329734 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:58.344772 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 630/1282] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Jul 25 19:12:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 19:12:58.352759 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d043b6a-5cf1-4279-a1bb-192d66711582 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:58.451963 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:58.517276 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d043b6a-5cf1-4279-a1bb-192d66711582 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:58.565088 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8abddae-a7d8-413a-9038-4b94927ccb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:58.566068 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abddae-a7d8-413a-9038-4b94927ccb86 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:58.566446 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abddae-a7d8-413a-9038-4b94927ccb86 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:58.566821 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8abddae-a7d8-413a-9038-4b94927ccb86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:58.585692 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8abddae-a7d8-413a-9038-4b94927ccb86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:58.588701 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8abddae-a7d8-413a-9038-4b94927ccb86 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:58.625621 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e8abddae-a7d8-413a-9038-4b94927ccb86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:12:58.625621 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:12:58.625621 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:12:58.625621 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:12:58.625621 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:12:58.625621 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:12:58.625621 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:12:58.625621 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:12:58.625621 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:12:58.625621 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:12:58.625621 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:12:58.625621 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:12:58.625621 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 19:12:58.625621 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 19:12:58.625621 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 19:12:58.626799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 19:12:58.626799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:12:58.626799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:12:58.626799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:12:58.626799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:12:58.626799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:12:58.626799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:12:58.626799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:12:58.626799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:12:58.626799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:12:58.626799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:12:58.626799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:12:58.626799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:12:58.626799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:12:58.626799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:12:58.626799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:12:58.626799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:12:58.628089 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 19:12:58.628089 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 19:12:58.628089 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 19:12:58.628089 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:12:58.628089 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 653/1283] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:12:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:12:58.638713 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d67f28-63db-4ef6-bc72-70f2ac093999 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:58.830623 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d67f28-63db-4ef6-bc72-70f2ac093999 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:58.874915 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98fd2fe-c3f2-4a60-88df-39f047cd3337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:58.875488 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98fd2fe-c3f2-4a60-88df-39f047cd3337 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:58.875802 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98fd2fe-c3f2-4a60-88df-39f047cd3337 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:58.876084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98fd2fe-c3f2-4a60-88df-39f047cd3337 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:58.876831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98fd2fe-c3f2-4a60-88df-39f047cd3337 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:12:58.877140 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98fd2fe-c3f2-4a60-88df-39f047cd3337 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:58.879332 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98fd2fe-c3f2-4a60-88df-39f047cd3337 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:58.896894 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 631/1284] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Jul 25 19:12:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:12:58.903344 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8abddae-a7d8-413a-9038-4b94927ccb86 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:59.033437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8abddae-a7d8-413a-9038-4b94927ccb86 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:59.051840 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:59.078472 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672b006-0011-4da3-856d-e8b25c952729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:59.078743 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672b006-0011-4da3-856d-e8b25c952729 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:59.079033 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672b006-0011-4da3-856d-e8b25c952729 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:59.079330 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672b006-0011-4da3-856d-e8b25c952729 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:59.089003 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b672b006-0011-4da3-856d-e8b25c952729 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:59.090650 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b672b006-0011-4da3-856d-e8b25c952729 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:59.107297 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b672b006-0011-4da3-856d-e8b25c952729 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:12:59.107297 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:12:59.107297 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:12:59.107297 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:12:59.107297 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:12:59.107297 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:12:59.107297 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:12:59.107297 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:12:59.107297 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:12:59.107297 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:12:59.107297 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:12:59.107297 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:12:59.107297 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 19:12:59.107297 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 19:12:59.107297 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 19:12:59.109121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 19:12:59.109121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:12:59.109121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:12:59.109121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:12:59.109121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:12:59.109121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:12:59.109121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:12:59.109121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:12:59.109121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jul 25 19:12:59.109121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:12:59.109121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:12:59.109121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:12:59.109121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:12:59.109121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:12:59.109121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:12:59.109121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:12:59.109121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:12:59.110930 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:12:59.110930 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:12:59.110930 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:12:59.110930 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:12:59.110930 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:12:59.110930 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:12:59.110930 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 19:12:59.110930 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jul 25 19:12:59.110930 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 654/1285] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Jul 25 19:12:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 19:12:59.116819 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98fd2fe-c3f2-4a60-88df-39f047cd3337 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:59.185901 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:12:59.253166 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98fd2fe-c3f2-4a60-88df-39f047cd3337 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:59.301399 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb54a7a6-a753-47ee-9af1-e66986caa126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:59.302487 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54a7a6-a753-47ee-9af1-e66986caa126 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:59.302990 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54a7a6-a753-47ee-9af1-e66986caa126 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:59.303478 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54a7a6-a753-47ee-9af1-e66986caa126 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:59.305076 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb54a7a6-a753-47ee-9af1-e66986caa126 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:12:59.305636 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb54a7a6-a753-47ee-9af1-e66986caa126 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:59.308578 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb54a7a6-a753-47ee-9af1-e66986caa126 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:59.332075 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 632/1286] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Jul 25 19:12:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:12:59.342080 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672b006-0011-4da3-856d-e8b25c952729 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:12:59.485317 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672b006-0011-4da3-856d-e8b25c952729 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:12:59.527857 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f595e68-cb0a-41bf-9d01-befa850e7d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:12:59.528536 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f595e68-cb0a-41bf-9d01-befa850e7d31 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:12:59.528820 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f595e68-cb0a-41bf-9d01-befa850e7d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:12:59.529103 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f595e68-cb0a-41bf-9d01-befa850e7d31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:12:59.529964 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f595e68-cb0a-41bf-9d01-befa850e7d31 None admin] RBAC: Adding query filter params (role.id=ed7fec18968241efa76a0ee769d96da9, scope.project.id=191069caccd04233b012585e29cfa18b, user.id=0422d6695bf14925bf0f993cb16e64ca) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:12:59.530281 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f595e68-cb0a-41bf-9d01-befa850e7d31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:12:59.531792 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f595e68-cb0a-41bf-9d01-befa850e7d31 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:12:59.540472 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 655/1287] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Jul 25 19:12:59 2022] GET /identity/v3/role_assignments?user.id=0422d6695bf14925bf0f993cb16e64ca&scope.project.id=191069caccd04233b012585e29cfa18b&role.id=ed7fec18968241efa76a0ee769d96da9 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:13:02.013439 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99631d-3467-4ff1-8eab-d0dd870d5f4a None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:13:02.014022 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99631d-3467-4ff1-8eab-d0dd870d5f4a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:13:02.014174 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99631d-3467-4ff1-8eab-d0dd870d5f4a None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:13:02.015818 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 633/1288] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:13:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:13:02.024483 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974780ef-b96f-4001-83c8-614b6eac755b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:13:02.025226 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974780ef-b96f-4001-83c8-614b6eac755b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:13:02.025376 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974780ef-b96f-4001-83c8-614b6eac755b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:13:02.053955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-974780ef-b96f-4001-83c8-614b6eac755b None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:13:02.055955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-974780ef-b96f-4001-83c8-614b6eac755b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:13:02.119682 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:13:02.180678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 656/1289] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:13:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:13:02.224919 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb99631d-3467-4ff1-8eab-d0dd870d5f4a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:13:02.424932 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb99631d-3467-4ff1-8eab-d0dd870d5f4a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:13:02.450434 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:13:02.486311 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c92c6d-b348-477c-8475-5eef073233f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:13:02.487281 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c92c6d-b348-477c-8475-5eef073233f7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:13:02.488206 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c92c6d-b348-477c-8475-5eef073233f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:13:02.488540 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c92c6d-b348-477c-8475-5eef073233f7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:13:02.489437 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c92c6d-b348-477c-8475-5eef073233f7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:13:02.491248 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c92c6d-b348-477c-8475-5eef073233f7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:13:02.510338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-22c92c6d-b348-477c-8475-5eef073233f7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 19:13:02.510338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:13:02.510338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 19:13:02.510338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 19:13:02.510338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 19:13:02.510338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 19:13:02.510338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 19:13:02.510338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:13:02.510338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 19:13:02.510338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:13:02.510338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 19:13:02.510338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:13:02.510338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:13:02.510338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:13:02.510338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:13:02.510338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:13:02.510338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:13:02.512711 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:13:02.512711 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:13:02.512711 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:13:02.512711 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:13:02.512711 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:13:02.512711 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:13:02.512711 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 19:13:02.512711 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 19:13:02.512711 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 19:13:02.512711 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 19:13:02.512711 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:13:02.512711 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:13:02.512711 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:13:02.512711 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:13:02.512711 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:13:02.512711 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:13:02.514414 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:13:02.514414 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:13:02.514414 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:13:02.514414 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:13:02.514414 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:13:02.514414 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:13:02.514414 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:13:02.514414 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:13:02.514414 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:13:02.514414 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:13:02.514414 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 19:13:02.514414 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:13:02.514414 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 19:13:02.514414 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:13:02.514414 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 634/1290] 10.222.0.18 () {56 vars in 1117 bytes} [Mon Jul 25 19:13:02 2022] GET /identity/v3/services/placement => generated 90 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:13:02.523558 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974780ef-b96f-4001-83c8-614b6eac755b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:13:02.747886 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974780ef-b96f-4001-83c8-614b6eac755b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:13:02.824936 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9c5a3b-ca92-4f64-9522-cd704cc502e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:13:02.826088 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c5a3b-ca92-4f64-9522-cd704cc502e4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:13:02.826605 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c5a3b-ca92-4f64-9522-cd704cc502e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:13:02.827098 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9c5a3b-ca92-4f64-9522-cd704cc502e4 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:13:02.828524 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9c5a3b-ca92-4f64-9522-cd704cc502e4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:13:02.829099 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9c5a3b-ca92-4f64-9522-cd704cc502e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:13:02.833593 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9c5a3b-ca92-4f64-9522-cd704cc502e4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:13:02.844082 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 657/1291] 10.222.0.18 () {56 vars in 1116 bytes} [Mon Jul 25 19:13:02 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:13:02.851471 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c92c6d-b348-477c-8475-5eef073233f7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:13:03.039773 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c92c6d-b348-477c-8475-5eef073233f7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:13:03.115590 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86626d-be41-44f6-a126-45d3db8488a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:13:03.116414 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86626d-be41-44f6-a126-45d3db8488a1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:13:03.116786 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86626d-be41-44f6-a126-45d3db8488a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:13:03.117129 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86626d-be41-44f6-a126-45d3db8488a1 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:13:03.117999 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe86626d-be41-44f6-a126-45d3db8488a1 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:13:03.118315 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe86626d-be41-44f6-a126-45d3db8488a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:13:03.119923 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe86626d-be41-44f6-a126-45d3db8488a1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:13:03.125819 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 635/1292] 10.222.0.18 () {56 vars in 1116 bytes} [Mon Jul 25 19:13:02 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:13:05.879272 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5817d1-06ed-4292-bb2a-9b57fb539531 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:13:05.880237 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5817d1-06ed-4292-bb2a-9b57fb539531 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:13:05.880394 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5817d1-06ed-4292-bb2a-9b57fb539531 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:13:05.882297 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 658/1293] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:13:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:13:05.891387 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26185e52-2f29-425a-9ce6-edecf781922e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:13:05.892002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26185e52-2f29-425a-9ce6-edecf781922e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:13:05.892002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26185e52-2f29-425a-9ce6-edecf781922e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:13:05.914751 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-26185e52-2f29-425a-9ce6-edecf781922e None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:13:05.916447 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-26185e52-2f29-425a-9ce6-edecf781922e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:13:05.970788 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:13:06.031977 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 636/1294] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:13:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:13:06.068441 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5817d1-06ed-4292-bb2a-9b57fb539531 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:13:06.129778 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:13:06.278298 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5817d1-06ed-4292-bb2a-9b57fb539531 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:13:06.326266 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f614805-8e76-4951-91e1-98c537cc3241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:13:06.327124 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f614805-8e76-4951-91e1-98c537cc3241 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:13:06.327460 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f614805-8e76-4951-91e1-98c537cc3241 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:13:06.327911 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f614805-8e76-4951-91e1-98c537cc3241 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:13:06.328969 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f614805-8e76-4951-91e1-98c537cc3241 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:13:06.330619 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f614805-8e76-4951-91e1-98c537cc3241 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:13:06.357555 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 659/1295] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Jul 25 19:13:06 2022] POST /identity/v3/services => generated 246 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:13:08.853928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377a7e7-459a-4ef1-aea1-3874a97f1788 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:13:08.854792 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377a7e7-459a-4ef1-aea1-3874a97f1788 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:13:08.854792 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377a7e7-459a-4ef1-aea1-3874a97f1788 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:13:08.856176 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 637/1296] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:13:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:13:08.862726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f988a-931d-49c6-b4d1-da140eedae93 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:13:08.863340 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f988a-931d-49c6-b4d1-da140eedae93 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:13:08.863340 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f988a-931d-49c6-b4d1-da140eedae93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:13:08.888823 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1f3f988a-931d-49c6-b4d1-da140eedae93 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:13:08.890879 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1f3f988a-931d-49c6-b4d1-da140eedae93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:13:09.000222 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 660/1297] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:13:08 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:13:09.026606 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c377a7e7-459a-4ef1-aea1-3874a97f1788 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:13:09.226770 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c377a7e7-459a-4ef1-aea1-3874a97f1788 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:13:09.278291 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce1a88e-abe6-46d8-84aa-49909c3f455f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:13:09.279188 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1a88e-abe6-46d8-84aa-49909c3f455f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:13:09.279530 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1a88e-abe6-46d8-84aa-49909c3f455f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:13:09.279866 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1a88e-abe6-46d8-84aa-49909c3f455f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:13:09.280595 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce1a88e-abe6-46d8-84aa-49909c3f455f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:13:09.282108 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce1a88e-abe6-46d8-84aa-49909c3f455f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:13:09.296799 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-8ce1a88e-abe6-46d8-84aa-49909c3f455f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 19:13:09.296799 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:13:09.296799 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 19:13:09.296799 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 19:13:09.296799 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 19:13:09.296799 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 19:13:09.296799 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 19:13:09.296799 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:13:09.296799 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 19:13:09.296799 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:13:09.296799 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 19:13:09.296799 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:13:09.296799 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:13:09.296799 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:13:09.296799 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:13:09.296799 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:13:09.296799 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:13:09.298461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:13:09.298461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:13:09.298461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:13:09.298461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:13:09.298461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:13:09.298461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:13:09.298461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 19:13:09.298461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 19:13:09.298461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 19:13:09.298461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 19:13:09.298461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:13:09.298461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:13:09.298461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:13:09.298461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:13:09.298461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:13:09.298461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:13:09.300040 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:13:09.300040 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:13:09.300040 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:13:09.300040 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:13:09.300040 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:13:09.300040 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:13:09.300040 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:13:09.300040 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:13:09.300040 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:13:09.300040 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:13:09.300040 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 19:13:09.300040 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:13:09.300040 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 19:13:09.300040 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:13:09.300040 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 638/1298] 10.222.0.18 () {56 vars in 1117 bytes} [Mon Jul 25 19:13:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:13:09.308411 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f988a-931d-49c6-b4d1-da140eedae93 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:13:09.332974 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:13:09.469602 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f988a-931d-49c6-b4d1-da140eedae93 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:13:09.531957 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1c21ef-236c-4d41-ba29-d88e86d3c175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:13:09.532809 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c21ef-236c-4d41-ba29-d88e86d3c175 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:13:09.533187 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c21ef-236c-4d41-ba29-d88e86d3c175 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:13:09.533564 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c21ef-236c-4d41-ba29-d88e86d3c175 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:13:09.535019 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1c21ef-236c-4d41-ba29-d88e86d3c175 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:13:09.535372 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1c21ef-236c-4d41-ba29-d88e86d3c175 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:13:09.537045 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1c21ef-236c-4d41-ba29-d88e86d3c175 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:13:09.543878 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 661/1299] 10.222.0.18 () {56 vars in 1116 bytes} [Mon Jul 25 19:13:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:13:09.555433 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce1a88e-abe6-46d8-84aa-49909c3f455f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:13:09.598767 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:13:09.712052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce1a88e-abe6-46d8-84aa-49909c3f455f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:13:09.760111 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e74bc-9870-4453-a6b6-0916d800f7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:13:09.760863 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e74bc-9870-4453-a6b6-0916d800f7ca None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:13:09.761268 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e74bc-9870-4453-a6b6-0916d800f7ca None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:13:09.761641 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e74bc-9870-4453-a6b6-0916d800f7ca None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:13:09.762621 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5e74bc-9870-4453-a6b6-0916d800f7ca None admin] RBAC: Adding query filter params (interface=public, service_id=a1f7417b29d54b97856d81acf4872db2, region_id=RegionOne) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:13:09.763046 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5e74bc-9870-4453-a6b6-0916d800f7ca None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:13:09.764698 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5e74bc-9870-4453-a6b6-0916d800f7ca None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:13:09.771934 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 639/1300] 10.222.0.18 () {56 vars in 1251 bytes} [Mon Jul 25 19:13:09 2022] GET /identity/v3/endpoints?service_id=a1f7417b29d54b97856d81acf4872db2&interface=public®ion_id=RegionOne => generated 194 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:13:09.780014 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1c21ef-236c-4d41-ba29-d88e86d3c175 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:13:09.909078 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:13:09.935799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1c21ef-236c-4d41-ba29-d88e86d3c175 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:13:09.986419 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0bb0d8-b72c-48de-8ab7-a4c192b5e099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:13:09.987406 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0bb0d8-b72c-48de-8ab7-a4c192b5e099 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:13:09.987737 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0bb0d8-b72c-48de-8ab7-a4c192b5e099 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:13:09.988107 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0bb0d8-b72c-48de-8ab7-a4c192b5e099 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:13:09.988827 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0bb0d8-b72c-48de-8ab7-a4c192b5e099 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:13:09.990249 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0bb0d8-b72c-48de-8ab7-a4c192b5e099 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:13:09.999124 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 662/1301] 10.222.0.18 () {56 vars in 1087 bytes} [Mon Jul 25 19:13:09 2022] GET /identity/v3/services => generated 2191 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:13:12.685991 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bf5a1-2578-40b8-86c9-77a3302f7fc7 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:13:12.686846 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bf5a1-2578-40b8-86c9-77a3302f7fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:13:12.687019 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bf5a1-2578-40b8-86c9-77a3302f7fc7 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:13:12.689725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 640/1302] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:13:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:13:12.699623 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9dc1f8-30e4-4eb6-940a-5b86db4933b3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:13:12.700256 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9dc1f8-30e4-4eb6-940a-5b86db4933b3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:13:12.700433 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9dc1f8-30e4-4eb6-940a-5b86db4933b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:13:12.732499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5e9dc1f8-30e4-4eb6-940a-5b86db4933b3 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:13:12.734565 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5e9dc1f8-30e4-4eb6-940a-5b86db4933b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:13:12.852858 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 663/1303] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:13:12 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 19:13:12.877931 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4bf5a1-2578-40b8-86c9-77a3302f7fc7 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:13:13.031915 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:13:13.042111 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4bf5a1-2578-40b8-86c9-77a3302f7fc7 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:13:13.084278 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bda204-1d6b-413b-8c0a-39497ed30910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:13:13.085198 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bda204-1d6b-413b-8c0a-39497ed30910 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:13:13.085483 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bda204-1d6b-413b-8c0a-39497ed30910 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:13:13.085859 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bda204-1d6b-413b-8c0a-39497ed30910 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:13:13.086533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bda204-1d6b-413b-8c0a-39497ed30910 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:13:13.088012 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bda204-1d6b-413b-8c0a-39497ed30910 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:13:13.102720 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-53bda204-1d6b-413b-8c0a-39497ed30910 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 19:13:13.102720 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:13:13.102720 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 19:13:13.102720 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 19:13:13.102720 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 19:13:13.102720 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 19:13:13.102720 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 19:13:13.102720 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:13:13.102720 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 19:13:13.102720 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:13:13.102720 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 19:13:13.102720 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:13:13.102720 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:13:13.102720 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:13:13.102720 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:13:13.102720 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:13:13.102720 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:13:13.105032 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:13:13.105032 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:13:13.105032 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:13:13.105032 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:13:13.105032 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:13:13.105032 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:13:13.105032 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 19:13:13.105032 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 19:13:13.105032 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 19:13:13.105032 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 19:13:13.105032 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:13:13.105032 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:13:13.105032 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:13:13.105032 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:13:13.105032 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:13:13.105032 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:13:13.106911 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:13:13.106911 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:13:13.106911 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:13:13.106911 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:13:13.106911 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:13:13.106911 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:13:13.106911 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:13:13.106911 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:13:13.106911 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:13:13.106911 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:13:13.106911 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 19:13:13.106911 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 19:13:13.106911 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 19:13:13.106911 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:13:13.106911 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 641/1304] 10.222.0.18 () {56 vars in 1117 bytes} [Mon Jul 25 19:13:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 19:13:13.114733 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9dc1f8-30e4-4eb6-940a-5b86db4933b3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:13:13.210885 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:13:13.239964 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9dc1f8-30e4-4eb6-940a-5b86db4933b3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:13:13.285755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e72b045-70d3-402c-ae59-240a294dcefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:13:13.286367 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72b045-70d3-402c-ae59-240a294dcefa None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:13:13.286814 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72b045-70d3-402c-ae59-240a294dcefa None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:13:13.286999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72b045-70d3-402c-ae59-240a294dcefa None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:13:13.287717 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e72b045-70d3-402c-ae59-240a294dcefa None admin] RBAC: Adding query filter params (name=placement) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:13:13.288032 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e72b045-70d3-402c-ae59-240a294dcefa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:13:13.289497 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e72b045-70d3-402c-ae59-240a294dcefa None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:13:13.295770 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 664/1305] 10.222.0.18 () {56 vars in 1116 bytes} [Mon Jul 25 19:13:13 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:13:13.306034 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bda204-1d6b-413b-8c0a-39497ed30910 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:13:13.444836 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bda204-1d6b-413b-8c0a-39497ed30910 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:13:13.491590 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d32a4-d3c4-4b56-a16d-f684e4bf1ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:13:13.492422 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d32a4-d3c4-4b56-a16d-f684e4bf1ec0 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:13:13.492585 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d32a4-d3c4-4b56-a16d-f684e4bf1ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:13:13.492855 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d32a4-d3c4-4b56-a16d-f684e4bf1ec0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:13:13.493745 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d32a4-d3c4-4b56-a16d-f684e4bf1ec0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:13:13.495363 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d32a4-d3c4-4b56-a16d-f684e4bf1ec0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:13:13.521368 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 642/1306] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:13:13 2022] POST /identity/v3/endpoints => generated 331 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:13:23.292614 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=30 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:13:23.515600 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:14:30.801008 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d016e8-5adb-445d-9be0-864511c47789 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:14:30.801839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d016e8-5adb-445d-9be0-864511c47789 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:14:30.801839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d016e8-5adb-445d-9be0-864511c47789 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:14:30.805336 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 665/1307] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:14:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:14:30.813149 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b81534-f043-4e72-85c6-ce9f7747f291 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:14:30.813945 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b81534-f043-4e72-85c6-ce9f7747f291 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:14:30.814422 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b81534-f043-4e72-85c6-ce9f7747f291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:14:30.929494 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-30b81534-f043-4e72-85c6-ce9f7747f291 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:14:30.931191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-30b81534-f043-4e72-85c6-ce9f7747f291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:14:31.128161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 643/1308] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:14:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:14:31.191798 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a34bb3-5794-41fd-a1ce-583f56ed7e6b None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:14:31.192352 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a34bb3-5794-41fd-a1ce-583f56ed7e6b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:14:31.192899 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a34bb3-5794-41fd-a1ce-583f56ed7e6b None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:14:31.196858 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 666/1309] 10.222.0.18 () {54 vars in 965 bytes} [Mon Jul 25 19:14:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:14:31.207935 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d6a9a-305e-4088-ab4a-3942224f2279 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:14:31.208432 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d6a9a-305e-4088-ab4a-3942224f2279 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:14:31.208843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d6a9a-305e-4088-ab4a-3942224f2279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:14:31.331310 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5a1d6a9a-305e-4088-ab4a-3942224f2279 None None] MFA Rules not processed for user `181c84715c094100b414cea3c4919eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:14:31.333321 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5a1d6a9a-305e-4088-ab4a-3942224f2279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:14:31.368737 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:14:31.465331 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 644/1310] 10.222.0.18 () {58 vars in 1062 bytes} [Mon Jul 25 19:14:31 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:14:31.481538 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a34bb3-5794-41fd-a1ce-583f56ed7e6b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:14:31.485344 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e5a34bb3-5794-41fd-a1ce-583f56ed7e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:14:31.661306 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a34bb3-5794-41fd-a1ce-583f56ed7e6b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:14:31.798823 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8175467-6153-4b6f-869e-7e9f97c54c5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:14:31.799911 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8175467-6153-4b6f-869e-7e9f97c54c5e service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:14:31.800212 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8175467-6153-4b6f-869e-7e9f97c54c5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:14:31.800718 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8175467-6153-4b6f-869e-7e9f97c54c5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:14:31.803227 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a8175467-6153-4b6f-869e-7e9f97c54c5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:14:31.883464 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:14:32.053107 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8175467-6153-4b6f-869e-7e9f97c54c5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:14:32.056377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8175467-6153-4b6f-869e-7e9f97c54c5e service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:14:32.101883 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 667/1311] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:14:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:14:35.091706 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3a451-b053-4bd9-a03b-60aa82a2a62c None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:14:35.092322 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3a451-b053-4bd9-a03b-60aa82a2a62c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:14:35.092322 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3a451-b053-4bd9-a03b-60aa82a2a62c None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:14:35.094346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 645/1312] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:14:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:14:35.100961 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f7a50-ca85-47e9-ac8f-a9ba82f6307d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:14:35.101747 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f7a50-ca85-47e9-ac8f-a9ba82f6307d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:14:35.101747 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f7a50-ca85-47e9-ac8f-a9ba82f6307d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:14:35.229760 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9a5f7a50-ca85-47e9-ac8f-a9ba82f6307d None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:14:35.231187 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9a5f7a50-ca85-47e9-ac8f-a9ba82f6307d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:14:35.410989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 668/1313] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:14:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:14:35.471989 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda3a451-b053-4bd9-a03b-60aa82a2a62c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:14:35.474882 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cda3a451-b053-4bd9-a03b-60aa82a2a62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:14:35.688741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda3a451-b053-4bd9-a03b-60aa82a2a62c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:14:35.793663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:14:35.798094 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3011e-49f1-4020-9e30-c003cdac8458 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:14:35.799946 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3011e-49f1-4020-9e30-c003cdac8458 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:14:35.800746 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3011e-49f1-4020-9e30-c003cdac8458 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:14:35.801790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3011e-49f1-4020-9e30-c003cdac8458 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:14:35.805464 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5da3011e-49f1-4020-9e30-c003cdac8458 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:14:36.046529 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da3011e-49f1-4020-9e30-c003cdac8458 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:14:36.048612 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da3011e-49f1-4020-9e30-c003cdac8458 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:14:36.080047 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 646/1314] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:14:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:14:38.915459 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a1cf2-dba3-4aad-bd74-4137d640d645 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:14:38.916247 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a1cf2-dba3-4aad-bd74-4137d640d645 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:14:38.916247 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a1cf2-dba3-4aad-bd74-4137d640d645 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:14:38.917969 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 669/1315] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:14:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:14:38.925432 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28837c-3338-4d8f-8932-22b462105e9e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:14:38.926046 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28837c-3338-4d8f-8932-22b462105e9e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:14:38.926046 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28837c-3338-4d8f-8932-22b462105e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:14:39.022262 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5c28837c-3338-4d8f-8932-22b462105e9e None None] MFA Rules not processed for user `1ee13b9555134b9cafac7f5f0a47204b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:14:39.023810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5c28837c-3338-4d8f-8932-22b462105e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:14:39.121705 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:14:39.196002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 647/1316] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:14:38 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:14:39.241598 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01a1cf2-dba3-4aad-bd74-4137d640d645 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:14:39.243180 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a01a1cf2-dba3-4aad-bd74-4137d640d645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:14:39.255326 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:14:39.416383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01a1cf2-dba3-4aad-bd74-4137d640d645 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:14:39.525236 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5157a4e3-80f6-4ebe-8607-4415b6f61e57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:14:39.526068 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5157a4e3-80f6-4ebe-8607-4415b6f61e57 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:14:39.526388 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5157a4e3-80f6-4ebe-8607-4415b6f61e57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:14:39.526630 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5157a4e3-80f6-4ebe-8607-4415b6f61e57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:14:39.528539 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5157a4e3-80f6-4ebe-8607-4415b6f61e57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:14:39.623495 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:14:39.696156 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5157a4e3-80f6-4ebe-8607-4415b6f61e57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:14:39.697643 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5157a4e3-80f6-4ebe-8607-4415b6f61e57 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:14:39.727570 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 670/1317] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:14:39 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:14:42.819249 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5daae6-3e58-4692-a0f1-ad22a3dcce84 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:14:42.820248 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5daae6-3e58-4692-a0f1-ad22a3dcce84 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:14:42.820248 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5daae6-3e58-4692-a0f1-ad22a3dcce84 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:14:42.821947 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 648/1318] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:14:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:14:42.830137 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414d77a-dab6-4b0d-aab1-892fd59018a3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:14:42.830961 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414d77a-dab6-4b0d-aab1-892fd59018a3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:14:42.830961 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414d77a-dab6-4b0d-aab1-892fd59018a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:14:42.927365 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7414d77a-dab6-4b0d-aab1-892fd59018a3 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:14:42.928707 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7414d77a-dab6-4b0d-aab1-892fd59018a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:14:43.062958 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 671/1319] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:14:42 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:14:43.107995 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5daae6-3e58-4692-a0f1-ad22a3dcce84 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:14:43.110536 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0e5daae6-3e58-4692-a0f1-ad22a3dcce84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:14:43.305662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5daae6-3e58-4692-a0f1-ad22a3dcce84 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:14:43.429411 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4fc79c-ec88-475c-8731-ab9a5af07404 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:14:43.430140 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4fc79c-ec88-475c-8731-ab9a5af07404 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:14:43.430513 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4fc79c-ec88-475c-8731-ab9a5af07404 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:14:43.430833 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4fc79c-ec88-475c-8731-ab9a5af07404 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:14:43.432868 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8c4fc79c-ec88-475c-8731-ab9a5af07404 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:14:43.444143 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:14:43.591309 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4fc79c-ec88-475c-8731-ab9a5af07404 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:14:43.595921 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4fc79c-ec88-475c-8731-ab9a5af07404 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:14:43.636544 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 649/1320] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:14:43 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:14:47.358956 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc56893-b886-4e61-983d-abe62f1c862a None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:14:47.359865 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc56893-b886-4e61-983d-abe62f1c862a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:14:47.360072 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc56893-b886-4e61-983d-abe62f1c862a None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:14:47.362828 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 672/1321] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:14:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:14:47.373252 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24dbe3-2e2e-4b5b-9356-deac1b87c89d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:14:47.373829 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24dbe3-2e2e-4b5b-9356-deac1b87c89d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:14:47.374271 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd24dbe3-2e2e-4b5b-9356-deac1b87c89d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:14:47.487312 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-fd24dbe3-2e2e-4b5b-9356-deac1b87c89d None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:14:47.489607 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fd24dbe3-2e2e-4b5b-9356-deac1b87c89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:14:47.599122 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:14:47.653795 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 650/1322] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:14:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:14:47.708054 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc56893-b886-4e61-983d-abe62f1c862a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:14:47.710006 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ecc56893-b886-4e61-983d-abe62f1c862a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:14:47.740869 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:14:47.869080 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc56893-b886-4e61-983d-abe62f1c862a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:14:47.979767 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a1d98-c536-4bb6-b103-c09991a8be87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:14:47.980533 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a1d98-c536-4bb6-b103-c09991a8be87 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:14:47.980867 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a1d98-c536-4bb6-b103-c09991a8be87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:14:47.981302 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a1d98-c536-4bb6-b103-c09991a8be87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:14:47.983175 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0a7a1d98-c536-4bb6-b103-c09991a8be87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:14:48.131871 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:14:48.202204 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7a1d98-c536-4bb6-b103-c09991a8be87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:14:48.205891 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7a1d98-c536-4bb6-b103-c09991a8be87 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:14:48.232920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 673/1323] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:14:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:14:50.794228 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a580bf-ab3d-471a-9b05-aefc934b911f None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:14:50.795097 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a580bf-ab3d-471a-9b05-aefc934b911f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:14:50.795097 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a580bf-ab3d-471a-9b05-aefc934b911f None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:14:50.797783 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 651/1324] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:14:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:14:50.804990 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe061d-7927-431f-bd75-5ef9456b9ec8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:14:50.805867 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe061d-7927-431f-bd75-5ef9456b9ec8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:14:50.805867 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe061d-7927-431f-bd75-5ef9456b9ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:14:50.925750 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-35fe061d-7927-431f-bd75-5ef9456b9ec8 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:14:50.927279 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-35fe061d-7927-431f-bd75-5ef9456b9ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:14:51.092910 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 674/1325] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:14:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:14:51.140954 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a580bf-ab3d-471a-9b05-aefc934b911f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:14:51.144609 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-12a580bf-ab3d-471a-9b05-aefc934b911f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:14:51.334860 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a580bf-ab3d-471a-9b05-aefc934b911f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:14:51.453632 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85371d82-248b-4caf-bef5-69666da9e809 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:14:51.454231 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85371d82-248b-4caf-bef5-69666da9e809 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:14:51.454537 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85371d82-248b-4caf-bef5-69666da9e809 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:14:51.455729 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85371d82-248b-4caf-bef5-69666da9e809 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:14:51.459154 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-85371d82-248b-4caf-bef5-69666da9e809 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:14:51.495416 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:14:51.641701 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85371d82-248b-4caf-bef5-69666da9e809 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:14:51.643562 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85371d82-248b-4caf-bef5-69666da9e809 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:14:51.681411 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 652/1326] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:14:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:14:54.336757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7afa7a-cf81-4290-b495-b4e49a9a13fa None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:14:54.337603 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7afa7a-cf81-4290-b495-b4e49a9a13fa None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:14:54.337818 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7afa7a-cf81-4290-b495-b4e49a9a13fa None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:14:54.339857 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 675/1327] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:14:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:14:54.348236 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b7892-254f-43d3-9b95-e49949ba34d3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:14:54.348987 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b7892-254f-43d3-9b95-e49949ba34d3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:14:54.349400 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b7892-254f-43d3-9b95-e49949ba34d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:14:54.492852 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8b9b7892-254f-43d3-9b95-e49949ba34d3 None None] MFA Rules not processed for user `1ee13b9555134b9cafac7f5f0a47204b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:14:54.494961 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8b9b7892-254f-43d3-9b95-e49949ba34d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:14:54.604217 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:14:54.650388 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 653/1328] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:14:54 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:14:54.727808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7afa7a-cf81-4290-b495-b4e49a9a13fa None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:14:54.729803 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5d7afa7a-cf81-4290-b495-b4e49a9a13fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:14:54.774599 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:14:54.922940 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7afa7a-cf81-4290-b495-b4e49a9a13fa None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:14:55.086255 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9b278-a897-4121-abec-8e9629009d07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:14:55.098337 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9b278-a897-4121-abec-8e9629009d07 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:14:55.098337 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9b278-a897-4121-abec-8e9629009d07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:14:55.098337 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9b278-a897-4121-abec-8e9629009d07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:14:55.098337 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6cf9b278-a897-4121-abec-8e9629009d07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:14:55.258269 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:14:55.330418 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf9b278-a897-4121-abec-8e9629009d07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:14:55.333413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf9b278-a897-4121-abec-8e9629009d07 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:14:55.369828 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 676/1329] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:14:54 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 642 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:14:58.052942 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6344a-faef-4765-b41a-2131d6fc6a06 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:14:58.053521 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6344a-faef-4765-b41a-2131d6fc6a06 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:14:58.053521 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6344a-faef-4765-b41a-2131d6fc6a06 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:14:58.054942 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 654/1330] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:14:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:14:58.062886 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377ebc6-c70b-4f98-8f78-c635d9177953 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:14:58.063637 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377ebc6-c70b-4f98-8f78-c635d9177953 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:14:58.063862 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377ebc6-c70b-4f98-8f78-c635d9177953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:14:58.170616 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9377ebc6-c70b-4f98-8f78-c635d9177953 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:14:58.172720 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9377ebc6-c70b-4f98-8f78-c635d9177953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:14:58.329939 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 677/1331] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:14:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:14:58.384317 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f6344a-faef-4765-b41a-2131d6fc6a06 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:14:58.388692 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a0f6344a-faef-4765-b41a-2131d6fc6a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:14:58.532505 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f6344a-faef-4765-b41a-2131d6fc6a06 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:14:58.668444 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a15f2-914b-4aa6-abe1-2ca13c8d234a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:14:58.669597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a15f2-914b-4aa6-abe1-2ca13c8d234a service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:14:58.670093 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a15f2-914b-4aa6-abe1-2ca13c8d234a service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:14:58.670599 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a15f2-914b-4aa6-abe1-2ca13c8d234a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:14:58.674720 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-166a15f2-914b-4aa6-abe1-2ca13c8d234a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:14:58.730506 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:14:58.855486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166a15f2-914b-4aa6-abe1-2ca13c8d234a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:14:58.857009 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166a15f2-914b-4aa6-abe1-2ca13c8d234a service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:14:58.880584 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 655/1332] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:14:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:15:01.126413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd559e-1624-4e91-ba5f-f9e8b21e94da None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:01.127052 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd559e-1624-4e91-ba5f-f9e8b21e94da None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:01.127052 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd559e-1624-4e91-ba5f-f9e8b21e94da None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:01.128506 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 678/1333] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:15:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:15:01.134716 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f890a29-1aee-492a-b8c7-fe403048cc3a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:01.135295 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f890a29-1aee-492a-b8c7-fe403048cc3a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:01.135295 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f890a29-1aee-492a-b8c7-fe403048cc3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:01.228613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6f890a29-1aee-492a-b8c7-fe403048cc3a None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:15:01.229872 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6f890a29-1aee-492a-b8c7-fe403048cc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:01.321855 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:01.350987 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 656/1334] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:15:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:15:01.413754 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1a166-b0d4-422c-b823-ee03f381ddc5 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:01.414201 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1a166-b0d4-422c-b823-ee03f381ddc5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:01.414609 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1a166-b0d4-422c-b823-ee03f381ddc5 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:01.416168 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 679/1335] 10.222.0.18 () {54 vars in 965 bytes} [Mon Jul 25 19:15:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:15:01.423916 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742572e-1684-47e9-9c40-4778872d3f85 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:01.424203 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742572e-1684-47e9-9c40-4778872d3f85 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:01.424495 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742572e-1684-47e9-9c40-4778872d3f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:01.540212 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6742572e-1684-47e9-9c40-4778872d3f85 None None] MFA Rules not processed for user `181c84715c094100b414cea3c4919eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:15:01.541470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6742572e-1684-47e9-9c40-4778872d3f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:01.647133 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 657/1336] 10.222.0.18 () {58 vars in 1062 bytes} [Mon Jul 25 19:15:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:15:01.658395 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1a166-b0d4-422c-b823-ee03f381ddc5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:15:01.661250 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0fc1a166-b0d4-422c-b823-ee03f381ddc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:01.712624 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:01.792670 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1a166-b0d4-422c-b823-ee03f381ddc5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:15:01.884260 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12434b4b-78e3-4804-b56b-04274cd48d69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:15:01.884931 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12434b4b-78e3-4804-b56b-04274cd48d69 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:01.885223 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12434b4b-78e3-4804-b56b-04274cd48d69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:01.885508 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12434b4b-78e3-4804-b56b-04274cd48d69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:01.887250 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-12434b4b-78e3-4804-b56b-04274cd48d69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:02.001779 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:02.053986 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12434b4b-78e3-4804-b56b-04274cd48d69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:15:02.056834 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12434b4b-78e3-4804-b56b-04274cd48d69 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:15:02.094918 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 680/1337] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:15:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:15:05.477001 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1e47a-cf44-4951-a40c-4ff6e9702d37 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:05.477856 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1e47a-cf44-4951-a40c-4ff6e9702d37 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:05.477856 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1e47a-cf44-4951-a40c-4ff6e9702d37 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:05.478886 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 658/1338] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:15:05.485599 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1460be-651c-4394-bf77-98fb1a222d6d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:05.486141 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1460be-651c-4394-bf77-98fb1a222d6d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:05.486251 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1460be-651c-4394-bf77-98fb1a222d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:05.568430 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-bb1460be-651c-4394-bf77-98fb1a222d6d None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:15:05.569743 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bb1460be-651c-4394-bf77-98fb1a222d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:05.707994 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 681/1339] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:15:05 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:15:05.750225 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a1e47a-cf44-4951-a40c-4ff6e9702d37 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:15:05.752037 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d9a1e47a-cf44-4951-a40c-4ff6e9702d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:05.855278 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:05.892393 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a1e47a-cf44-4951-a40c-4ff6e9702d37 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:15:06.010609 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa44f2-a2ef-4ace-8b74-44a8e9e39c2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:15:06.011855 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa44f2-a2ef-4ace-8b74-44a8e9e39c2b service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:06.012231 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa44f2-a2ef-4ace-8b74-44a8e9e39c2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:06.012725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa44f2-a2ef-4ace-8b74-44a8e9e39c2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:06.018101 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ebfa44f2-a2ef-4ace-8b74-44a8e9e39c2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:06.223525 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfa44f2-a2ef-4ace-8b74-44a8e9e39c2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:15:06.225293 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfa44f2-a2ef-4ace-8b74-44a8e9e39c2b service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:15:06.253695 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 659/1340] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:15:05 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:15:11.430791 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4112638-4db1-45ee-98df-a74032974d11 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:11.431624 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4112638-4db1-45ee-98df-a74032974d11 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:11.431624 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4112638-4db1-45ee-98df-a74032974d11 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:11.441219 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b2abd6-c9bc-4fa2-834f-bb30665d9d05 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:11.442105 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b2abd6-c9bc-4fa2-834f-bb30665d9d05 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:11.442278 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b2abd6-c9bc-4fa2-834f-bb30665d9d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:11.444234 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 682/1341] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:15:11 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:15:11.454543 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:11.577243 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f9b2abd6-c9bc-4fa2-834f-bb30665d9d05 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:15:11.578967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f9b2abd6-c9bc-4fa2-834f-bb30665d9d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:11.742979 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 660/1342] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:15:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:15:11.799671 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4112638-4db1-45ee-98df-a74032974d11 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:15:11.802592 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b4112638-4db1-45ee-98df-a74032974d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:11.886649 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:12.003196 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4112638-4db1-45ee-98df-a74032974d11 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:15:12.135152 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fde658-f6d0-4e5c-a0ac-050ba7b869d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:15:12.136373 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fde658-f6d0-4e5c-a0ac-050ba7b869d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:12.136887 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fde658-f6d0-4e5c-a0ac-050ba7b869d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:12.137554 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fde658-f6d0-4e5c-a0ac-050ba7b869d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:12.141734 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-83fde658-f6d0-4e5c-a0ac-050ba7b869d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:12.318912 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:12.352313 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fde658-f6d0-4e5c-a0ac-050ba7b869d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:15:12.354972 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fde658-f6d0-4e5c-a0ac-050ba7b869d1 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:15:12.392227 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 683/1343] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:15:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:15:16.448958 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486bd8a1-97ee-40e4-92dd-9d0435446e3d None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:16.449607 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486bd8a1-97ee-40e4-92dd-9d0435446e3d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:16.449833 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486bd8a1-97ee-40e4-92dd-9d0435446e3d None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:16.453700 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 661/1344] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:15:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:15:16.462716 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a07678-8d7b-4c20-8acf-5f0703c0217f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:16.463378 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a07678-8d7b-4c20-8acf-5f0703c0217f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:16.463599 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a07678-8d7b-4c20-8acf-5f0703c0217f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:16.599821 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-08a07678-8d7b-4c20-8acf-5f0703c0217f None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:15:16.602036 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-08a07678-8d7b-4c20-8acf-5f0703c0217f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:16.781979 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 684/1345] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:15:16 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:15:16.831336 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486bd8a1-97ee-40e4-92dd-9d0435446e3d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:15:16.834380 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-486bd8a1-97ee-40e4-92dd-9d0435446e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:16.969564 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:17.003916 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486bd8a1-97ee-40e4-92dd-9d0435446e3d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:15:17.110656 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41e9062-f75e-46a2-a93d-b3a5abcc382f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:15:17.111329 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e9062-f75e-46a2-a93d-b3a5abcc382f service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:17.111641 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e9062-f75e-46a2-a93d-b3a5abcc382f service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:17.111919 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e9062-f75e-46a2-a93d-b3a5abcc382f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:17.113733 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a41e9062-f75e-46a2-a93d-b3a5abcc382f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:17.300478 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41e9062-f75e-46a2-a93d-b3a5abcc382f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:15:17.302402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41e9062-f75e-46a2-a93d-b3a5abcc382f service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:15:17.333531 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 662/1346] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:15:16 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:15:22.572613 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c249917-f903-4b73-8a46-7fc33de271b9 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:22.574190 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c249917-f903-4b73-8a46-7fc33de271b9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:22.574597 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c249917-f903-4b73-8a46-7fc33de271b9 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:22.577937 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 685/1347] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:15:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:15:22.587433 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749b8bf-c3e1-46fd-bb26-6f0a861a158a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:22.588542 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749b8bf-c3e1-46fd-bb26-6f0a861a158a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:22.589217 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749b8bf-c3e1-46fd-bb26-6f0a861a158a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:22.622688 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:22.716691 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f749b8bf-c3e1-46fd-bb26-6f0a861a158a None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:15:22.719005 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f749b8bf-c3e1-46fd-bb26-6f0a861a158a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:22.873004 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 663/1348] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:15:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:15:22.916229 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c249917-f903-4b73-8a46-7fc33de271b9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:15:22.917815 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9c249917-f903-4b73-8a46-7fc33de271b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:23.001455 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:23.064146 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c249917-f903-4b73-8a46-7fc33de271b9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:15:23.165939 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f37b7-8e1c-4fdf-a65a-c2e03678956d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:15:23.166609 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f37b7-8e1c-4fdf-a65a-c2e03678956d service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:23.166896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f37b7-8e1c-4fdf-a65a-c2e03678956d service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:23.167173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f37b7-8e1c-4fdf-a65a-c2e03678956d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:23.169597 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-744f37b7-8e1c-4fdf-a65a-c2e03678956d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:23.308586 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:23.313984 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f37b7-8e1c-4fdf-a65a-c2e03678956d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:15:23.315488 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f37b7-8e1c-4fdf-a65a-c2e03678956d service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:15:23.345063 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 686/1349] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:15:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:15:29.423587 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17719f-e35d-4432-a826-77832aa13e93 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:29.424341 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17719f-e35d-4432-a826-77832aa13e93 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:29.424341 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17719f-e35d-4432-a826-77832aa13e93 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:29.427146 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 664/1350] 10.222.0.18 () {54 vars in 921 bytes} [Mon Jul 25 19:15:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:15:29.432702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d58b4-950a-47e0-99d2-40898801f8c6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:29.433506 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d58b4-950a-47e0-99d2-40898801f8c6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:29.433506 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d58b4-950a-47e0-99d2-40898801f8c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:29.599496 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-644d58b4-950a-47e0-99d2-40898801f8c6 None None] MFA Rules not processed for user `0422d6695bf14925bf0f993cb16e64ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:15:29.610896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-644d58b4-950a-47e0-99d2-40898801f8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:29.831934 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 687/1351] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Jul 25 19:15:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:15:32.869167 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=59 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:35.413004 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c023feb9-df4e-47d7-a642-557585fa8a1e None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:35.414002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c023feb9-df4e-47d7-a642-557585fa8a1e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:35.414002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c023feb9-df4e-47d7-a642-557585fa8a1e None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:35.415237 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 665/1352] 10.222.0.18 () {54 vars in 921 bytes} [Mon Jul 25 19:15:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:15:35.425926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95beb0b-e20a-41a0-baeb-33a8194953fd None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:35.426748 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95beb0b-e20a-41a0-baeb-33a8194953fd None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:35.427172 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95beb0b-e20a-41a0-baeb-33a8194953fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:35.564352 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:35.608505 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f95beb0b-e20a-41a0-baeb-33a8194953fd None None] MFA Rules not processed for user `0422d6695bf14925bf0f993cb16e64ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:15:35.611122 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f95beb0b-e20a-41a0-baeb-33a8194953fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:35.866397 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 688/1353] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Jul 25 19:15:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 442 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:15:37.129164 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da2c0c-1714-4dff-abac-4311c92e72af None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:37.129813 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da2c0c-1714-4dff-abac-4311c92e72af None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:37.129813 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da2c0c-1714-4dff-abac-4311c92e72af None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:37.131354 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 666/1354] 10.222.0.18 () {54 vars in 921 bytes} [Mon Jul 25 19:15:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:15:37.137239 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5228b2-53e8-43f3-8456-c650be285032 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:37.138054 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5228b2-53e8-43f3-8456-c650be285032 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:37.138054 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5228b2-53e8-43f3-8456-c650be285032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:37.250590 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8e5228b2-53e8-43f3-8456-c650be285032 None None] MFA Rules not processed for user `0422d6695bf14925bf0f993cb16e64ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:15:37.252040 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8e5228b2-53e8-43f3-8456-c650be285032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:37.325379 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:37.451798 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 689/1355] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Jul 25 19:15:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:15:41.303244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937a3f7-cfc3-422b-99d1-776d1256c8d3 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:41.303979 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937a3f7-cfc3-422b-99d1-776d1256c8d3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:41.304247 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937a3f7-cfc3-422b-99d1-776d1256c8d3 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:41.306825 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 667/1356] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:15:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:15:41.316473 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd9c332-a334-4c0e-b10a-1a5902fc4bac None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:41.317370 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd9c332-a334-4c0e-b10a-1a5902fc4bac None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:41.317977 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd9c332-a334-4c0e-b10a-1a5902fc4bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:41.423295 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6fd9c332-a334-4c0e-b10a-1a5902fc4bac None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:15:41.425149 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6fd9c332-a334-4c0e-b10a-1a5902fc4bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:41.671620 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 690/1357] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:15:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:15:41.735684 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6199c6-19f4-47e3-8afc-835ecb0f10d4 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:41.736263 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6199c6-19f4-47e3-8afc-835ecb0f10d4 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:41.736745 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6199c6-19f4-47e3-8afc-835ecb0f10d4 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:41.739414 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 668/1358] 10.222.0.18 () {54 vars in 962 bytes} [Mon Jul 25 19:15:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:15:41.749820 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e5cbd-da05-4684-bb06-78ac87332fa4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:41.750187 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e5cbd-da05-4684-bb06-78ac87332fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:41.750652 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e5cbd-da05-4684-bb06-78ac87332fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:41.772024 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:41.875309 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-bc7e5cbd-da05-4684-bb06-78ac87332fa4 None None] MFA Rules not processed for user `860e7d26ad364795bb25836aadd4ac7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:15:41.877215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bc7e5cbd-da05-4684-bb06-78ac87332fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:42.031917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 691/1359] 10.222.0.18 () {58 vars in 1059 bytes} [Mon Jul 25 19:15:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:15:42.039118 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6199c6-19f4-47e3-8afc-835ecb0f10d4 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:15:42.041191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3f6199c6-19f4-47e3-8afc-835ecb0f10d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:42.320025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6199c6-19f4-47e3-8afc-835ecb0f10d4 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:15:42.474642 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:42.540424 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa969b5-e348-48a1-80e6-23b874ad1e64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:15:42.541790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa969b5-e348-48a1-80e6-23b874ad1e64 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:42.542383 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa969b5-e348-48a1-80e6-23b874ad1e64 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:42.542934 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa969b5-e348-48a1-80e6-23b874ad1e64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:42.546558 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cfa969b5-e348-48a1-80e6-23b874ad1e64 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:42.691782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa969b5-e348-48a1-80e6-23b874ad1e64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:15:42.693814 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa969b5-e348-48a1-80e6-23b874ad1e64 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:15:42.723191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 669/1360] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:15:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 685 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:15:45.584448 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8093f404-070c-48a1-a950-7ef50839d136 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:45.585320 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8093f404-070c-48a1-a950-7ef50839d136 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:45.585320 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8093f404-070c-48a1-a950-7ef50839d136 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:45.587424 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 692/1361] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:15:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:15:45.594559 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a1576-75e8-4189-8ea8-4ded9e2dcf07 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:45.595697 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a1576-75e8-4189-8ea8-4ded9e2dcf07 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:45.595697 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a1576-75e8-4189-8ea8-4ded9e2dcf07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:45.710701 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0b4a1576-75e8-4189-8ea8-4ded9e2dcf07 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:15:45.712580 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0b4a1576-75e8-4189-8ea8-4ded9e2dcf07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:45.771223 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:45.873453 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 670/1362] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:15:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:15:45.946469 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8093f404-070c-48a1-a950-7ef50839d136 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:15:45.948149 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8093f404-070c-48a1-a950-7ef50839d136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:46.053856 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:46.255411 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8093f404-070c-48a1-a950-7ef50839d136 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:15:46.538879 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb0c496-7e5e-45fa-b28f-1975f216111a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:15:46.540472 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0c496-7e5e-45fa-b28f-1975f216111a service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:46.541031 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0c496-7e5e-45fa-b28f-1975f216111a service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:46.541529 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb0c496-7e5e-45fa-b28f-1975f216111a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:46.546294 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ceb0c496-7e5e-45fa-b28f-1975f216111a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:46.595089 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:46.712583 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb0c496-7e5e-45fa-b28f-1975f216111a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:15:46.714188 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb0c496-7e5e-45fa-b28f-1975f216111a service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:15:46.748138 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 693/1363] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:15:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 802 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:15:50.243133 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e6288-b626-4936-8156-c32d49b59893 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:50.244085 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e6288-b626-4936-8156-c32d49b59893 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:50.244085 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e6288-b626-4936-8156-c32d49b59893 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:50.246832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 671/1364] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:15:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:15:50.253314 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75437f-479a-4445-8e4f-b394a1939bb4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:50.254160 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75437f-479a-4445-8e4f-b394a1939bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:50.254307 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75437f-479a-4445-8e4f-b394a1939bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:50.369136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7b75437f-479a-4445-8e4f-b394a1939bb4 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:15:50.370579 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7b75437f-479a-4445-8e4f-b394a1939bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:50.530756 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:50.546503 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 694/1365] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:15:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:15:50.586826 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ccb8d-b627-4f8b-8d9d-014778669177 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:50.588192 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ccb8d-b627-4f8b-8d9d-014778669177 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:50.588636 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ccb8d-b627-4f8b-8d9d-014778669177 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:50.590793 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 672/1366] 10.222.0.18 () {54 vars in 962 bytes} [Mon Jul 25 19:15:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:15:50.598164 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c6918-4dd1-4b55-91cb-b9092a455d0d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:50.598469 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c6918-4dd1-4b55-91cb-b9092a455d0d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:50.598888 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c6918-4dd1-4b55-91cb-b9092a455d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:50.711236 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c53c6918-4dd1-4b55-91cb-b9092a455d0d None None] MFA Rules not processed for user `860e7d26ad364795bb25836aadd4ac7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:15:50.712747 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c53c6918-4dd1-4b55-91cb-b9092a455d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:50.892013 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 695/1367] 10.222.0.18 () {58 vars in 1059 bytes} [Mon Jul 25 19:15:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:15:50.899345 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ccb8d-b627-4f8b-8d9d-014778669177 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:15:50.901019 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e95ccb8d-b627-4f8b-8d9d-014778669177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:51.163404 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ccb8d-b627-4f8b-8d9d-014778669177 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:15:51.211600 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:51.439092 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0edb60-94f2-4db1-b248-983400b4bda3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:15:51.441804 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0edb60-94f2-4db1-b248-983400b4bda3 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:51.441804 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0edb60-94f2-4db1-b248-983400b4bda3 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:51.441804 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0edb60-94f2-4db1-b248-983400b4bda3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:51.443905 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6d0edb60-94f2-4db1-b248-983400b4bda3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:51.493149 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7562fa2-431a-4b61-b9c9-bb77f57b54d7 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:51.494129 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7562fa2-431a-4b61-b9c9-bb77f57b54d7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:51.494129 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7562fa2-431a-4b61-b9c9-bb77f57b54d7 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:51.496763 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 696/1368] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:15:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:15:51.506182 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a017a6-4826-4ad0-95d0-8742fb2c61e3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:51.506630 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a017a6-4826-4ad0-95d0-8742fb2c61e3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:51.507102 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a017a6-4826-4ad0-95d0-8742fb2c61e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:51.615360 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0edb60-94f2-4db1-b248-983400b4bda3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:15:51.618302 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0edb60-94f2-4db1-b248-983400b4bda3 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:15:51.632485 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:51.657324 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-93a017a6-4826-4ad0-95d0-8742fb2c61e3 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:15:51.660969 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-93a017a6-4826-4ad0-95d0-8742fb2c61e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:51.663893 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 673/1369] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:15:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 765 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:15:51.887908 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 697/1370] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:15:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:15:52.293046 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5e41c-c0e7-4263-b996-edf08dc2c71d None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:52.293046 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5e41c-c0e7-4263-b996-edf08dc2c71d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:52.293046 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5e41c-c0e7-4263-b996-edf08dc2c71d None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:52.301933 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 674/1371] 10.222.0.18 () {54 vars in 964 bytes} [Mon Jul 25 19:15:52 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:15:52.303487 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f0546-cbdf-4547-9498-e0d782e0a5c9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:52.303487 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f0546-cbdf-4547-9498-e0d782e0a5c9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:52.303487 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f0546-cbdf-4547-9498-e0d782e0a5c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:52.420191 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1d1f0546-cbdf-4547-9498-e0d782e0a5c9 None None] MFA Rules not processed for user `2aa720a35b9844848abcc42a4c4d6a2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:15:52.421827 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1d1f0546-cbdf-4547-9498-e0d782e0a5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:52.554005 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:52.571794 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 698/1372] 10.222.0.18 () {58 vars in 1061 bytes} [Mon Jul 25 19:15:52 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:15:52.580554 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f5e41c-c0e7-4263-b996-edf08dc2c71d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:15:52.583963 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f9f5e41c-c0e7-4263-b996-edf08dc2c71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:52.648658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:52.772292 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f5e41c-c0e7-4263-b996-edf08dc2c71d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:15:52.905676 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a755088-ff8b-4d65-a7dc-865b389cfa7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:15:52.906670 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a755088-ff8b-4d65-a7dc-865b389cfa7d service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:52.907154 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a755088-ff8b-4d65-a7dc-865b389cfa7d service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:52.907679 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a755088-ff8b-4d65-a7dc-865b389cfa7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:52.910859 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9a755088-ff8b-4d65-a7dc-865b389cfa7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:53.126609 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:53.188600 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a755088-ff8b-4d65-a7dc-865b389cfa7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:15:53.191191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a755088-ff8b-4d65-a7dc-865b389cfa7d service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:15:53.237206 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 675/1373] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:15:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 658 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:15:54.188217 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8c140-28e0-4066-b72a-0d2364367ae3 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:54.188902 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8c140-28e0-4066-b72a-0d2364367ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:54.188902 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8c140-28e0-4066-b72a-0d2364367ae3 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:54.190151 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 699/1374] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:15:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:15:54.197673 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b59c4b-fecc-4a4a-bc82-ed6d908cce68 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:54.198283 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b59c4b-fecc-4a4a-bc82-ed6d908cce68 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:54.198923 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b59c4b-fecc-4a4a-bc82-ed6d908cce68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:54.308469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f8b59c4b-fecc-4a4a-bc82-ed6d908cce68 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:15:54.309742 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f8b59c4b-fecc-4a4a-bc82-ed6d908cce68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:54.438397 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 676/1375] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:15:54.488537 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc8c140-28e0-4066-b72a-0d2364367ae3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:15:54.490455 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ddc8c140-28e0-4066-b72a-0d2364367ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:54.685874 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc8c140-28e0-4066-b72a-0d2364367ae3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:15:54.808131 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:54.916443 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f8a4b9-1d8c-4a73-be01-6944c6000560 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:15:54.917556 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8a4b9-1d8c-4a73-be01-6944c6000560 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:54.918025 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8a4b9-1d8c-4a73-be01-6944c6000560 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:54.918510 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8a4b9-1d8c-4a73-be01-6944c6000560 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:54.922494 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-56f8a4b9-1d8c-4a73-be01-6944c6000560 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:55.065407 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f8a4b9-1d8c-4a73-be01-6944c6000560 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:15:55.067319 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f8a4b9-1d8c-4a73-be01-6944c6000560 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:15:55.101553 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 700/1376] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:15:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:15:57.887763 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ae75e-a1a8-47c9-9906-e8e3186e2421 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:57.888615 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ae75e-a1a8-47c9-9906-e8e3186e2421 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:57.888615 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ae75e-a1a8-47c9-9906-e8e3186e2421 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:57.890880 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 677/1377] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:15:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:15:57.897437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad4f30-c4c6-4b99-ad97-9daa5f103369 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:57.898285 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad4f30-c4c6-4b99-ad97-9daa5f103369 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:57.898285 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad4f30-c4c6-4b99-ad97-9daa5f103369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:57.991816 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c7ad4f30-c4c6-4b99-ad97-9daa5f103369 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:15:57.993422 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c7ad4f30-c4c6-4b99-ad97-9daa5f103369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:58.068138 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:58.192295 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 701/1378] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:15:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:15:58.269501 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ae75e-a1a8-47c9-9906-e8e3186e2421 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:15:58.272441 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-284ae75e-a1a8-47c9-9906-e8e3186e2421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:58.352069 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:58.576948 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ae75e-a1a8-47c9-9906-e8e3186e2421 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:15:58.794725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:15:58.808813 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307cbec0-fde2-494f-a8aa-77ee5fcb8942 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:15:58.808813 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307cbec0-fde2-494f-a8aa-77ee5fcb8942 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:15:58.808813 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307cbec0-fde2-494f-a8aa-77ee5fcb8942 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:15:58.808813 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307cbec0-fde2-494f-a8aa-77ee5fcb8942 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:15:58.811650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-307cbec0-fde2-494f-a8aa-77ee5fcb8942 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:15:58.960377 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307cbec0-fde2-494f-a8aa-77ee5fcb8942 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:15:58.963878 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307cbec0-fde2-494f-a8aa-77ee5fcb8942 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:15:58.994356 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 678/1379] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:15:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 726 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:16:01.901931 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b2389-2190-43b1-9ec8-cd3204210188 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:01.902780 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b2389-2190-43b1-9ec8-cd3204210188 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:01.902780 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b2389-2190-43b1-9ec8-cd3204210188 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:01.905626 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 702/1380] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:16:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:16:01.913600 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7d5d0-6502-448b-9d7b-ac6a80b80bdc None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:01.914487 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7d5d0-6502-448b-9d7b-ac6a80b80bdc None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:01.914487 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7d5d0-6502-448b-9d7b-ac6a80b80bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:02.055988 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a0f7d5d0-6502-448b-9d7b-ac6a80b80bdc None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:02.058682 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a0f7d5d0-6502-448b-9d7b-ac6a80b80bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:02.224354 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:02.301794 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 679/1381] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:16:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:16:02.385749 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b2389-2190-43b1-9ec8-cd3204210188 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:02.388932 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b11b2389-2190-43b1-9ec8-cd3204210188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:02.707110 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b2389-2190-43b1-9ec8-cd3204210188 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:02.728548 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:02.879383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279fcec-e9a9-4139-95e0-85189817f55c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:02.880156 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279fcec-e9a9-4139-95e0-85189817f55c service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:02.880576 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279fcec-e9a9-4139-95e0-85189817f55c service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:02.880948 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279fcec-e9a9-4139-95e0-85189817f55c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:02.883077 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4279fcec-e9a9-4139-95e0-85189817f55c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:03.015767 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4279fcec-e9a9-4139-95e0-85189817f55c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:03.018547 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4279fcec-e9a9-4139-95e0-85189817f55c service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:03.046415 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 703/1382] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:16:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 662 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:16:05.927908 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6febcb-9eda-4e50-872d-ba991148482f None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:05.928809 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6febcb-9eda-4e50-872d-ba991148482f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:05.928981 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6febcb-9eda-4e50-872d-ba991148482f None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:05.930944 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 680/1383] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:16:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:16:05.939009 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443e24c-d8ec-48e2-aab9-616ab0c40595 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:05.939926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443e24c-d8ec-48e2-aab9-616ab0c40595 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:05.940266 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443e24c-d8ec-48e2-aab9-616ab0c40595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:05.980620 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:06.081414 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f443e24c-d8ec-48e2-aab9-616ab0c40595 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:06.083458 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f443e24c-d8ec-48e2-aab9-616ab0c40595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:06.281140 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 704/1384] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:16:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:06.344040 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6febcb-9eda-4e50-872d-ba991148482f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:06.347115 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ce6febcb-9eda-4e50-872d-ba991148482f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:06.625380 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6febcb-9eda-4e50-872d-ba991148482f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:06.741440 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:06.843062 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c59e9-346b-498d-affe-e932eecc1bc2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:06.844055 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c59e9-346b-498d-affe-e932eecc1bc2 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:06.844385 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c59e9-346b-498d-affe-e932eecc1bc2 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:06.844706 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c59e9-346b-498d-affe-e932eecc1bc2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:06.847925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6e2c59e9-346b-498d-affe-e932eecc1bc2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:07.001854 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2c59e9-346b-498d-affe-e932eecc1bc2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:07.003909 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2c59e9-346b-498d-affe-e932eecc1bc2 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:07.033898 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 681/1385] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:16:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 692 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:16:09.774177 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8268bbe-026f-4252-8532-6299840d30e0 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:09.775186 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8268bbe-026f-4252-8532-6299840d30e0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:09.775186 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8268bbe-026f-4252-8532-6299840d30e0 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:09.778131 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 705/1386] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:16:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:16:09.788325 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31532b4a-ff5c-4036-a16d-2d115d5efa72 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:09.789507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31532b4a-ff5c-4036-a16d-2d115d5efa72 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:09.789507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31532b4a-ff5c-4036-a16d-2d115d5efa72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:09.920730 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-31532b4a-ff5c-4036-a16d-2d115d5efa72 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:09.923030 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-31532b4a-ff5c-4036-a16d-2d115d5efa72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:09.937728 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:10.129498 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 682/1387] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:16:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:10.189578 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8268bbe-026f-4252-8532-6299840d30e0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:10.193110 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c8268bbe-026f-4252-8532-6299840d30e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:10.447884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:10.560601 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8268bbe-026f-4252-8532-6299840d30e0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:10.758744 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a739f-9de7-488b-8fba-60d7948ae891 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:10.759337 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a739f-9de7-488b-8fba-60d7948ae891 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:10.759667 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a739f-9de7-488b-8fba-60d7948ae891 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:10.759954 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a739f-9de7-488b-8fba-60d7948ae891 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:10.761817 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e91a739f-9de7-488b-8fba-60d7948ae891 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:10.845457 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:10.904541 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91a739f-9de7-488b-8fba-60d7948ae891 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:10.906233 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91a739f-9de7-488b-8fba-60d7948ae891 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:10.930166 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 706/1388] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:16:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 742 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:16:13.630786 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0e83d-2492-434c-b048-d48492382b7e None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:13.631565 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0e83d-2492-434c-b048-d48492382b7e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:13.631784 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0e83d-2492-434c-b048-d48492382b7e None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:13.634453 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 683/1389] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:16:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:16:13.643275 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5209a9-b9b9-4f9c-a547-a36d06518944 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:13.643989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5209a9-b9b9-4f9c-a547-a36d06518944 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:13.643989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5209a9-b9b9-4f9c-a547-a36d06518944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:13.717445 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff564e-34c8-469c-8dfc-df7565c04dbb None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:13.717765 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff564e-34c8-469c-8dfc-df7565c04dbb None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:13.718002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff564e-34c8-469c-8dfc-df7565c04dbb None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:13.719695 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 684/1390] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:16:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:16:13.725846 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d391c7-3696-4858-8912-c218f2b26cbe None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:13.726080 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d391c7-3696-4858-8912-c218f2b26cbe None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:13.726283 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d391c7-3696-4858-8912-c218f2b26cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:13.760135 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4d5209a9-b9b9-4f9c-a547-a36d06518944 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:13.761471 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4d5209a9-b9b9-4f9c-a547-a36d06518944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:13.839697 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e6d391c7-3696-4858-8912-c218f2b26cbe None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:13.842225 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e6d391c7-3696-4858-8912-c218f2b26cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:13.925349 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 707/1391] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:16:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:13.986415 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:14.004195 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5209a9-b9b9-4f9c-a547-a36d06518944 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:14.005657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4d5209a9-b9b9-4f9c-a547-a36d06518944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:14.007500 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 685/1392] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:16:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:16:14.080723 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:14.081593 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8910e-92d6-4416-93a5-eea089f244e2 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:14.082083 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8910e-92d6-4416-93a5-eea089f244e2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:14.082584 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8910e-92d6-4416-93a5-eea089f244e2 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:14.084818 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 686/1393] 10.222.0.18 () {54 vars in 966 bytes} [Mon Jul 25 19:16:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:16:14.092469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b293991-a210-4e5c-bfb3-5a598eb2998f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:14.092836 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b293991-a210-4e5c-bfb3-5a598eb2998f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:14.093274 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b293991-a210-4e5c-bfb3-5a598eb2998f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:14.226049 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5209a9-b9b9-4f9c-a547-a36d06518944 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:14.234463 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4b293991-a210-4e5c-bfb3-5a598eb2998f None None] MFA Rules not processed for user `98560723b49940d38a6a5e9cf3f5d3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:14.236614 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4b293991-a210-4e5c-bfb3-5a598eb2998f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:14.373559 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 687/1394] 10.222.0.18 () {58 vars in 1063 bytes} [Mon Jul 25 19:16:14 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:16:14.384914 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b293991-a210-4e5c-bfb3-5a598eb2998f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:14.400432 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b293991-a210-4e5c-bfb3-5a598eb2998f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:14.415913 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa0568c-834f-48a5-ba39-6ff968207442 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:14.417033 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0568c-834f-48a5-ba39-6ff968207442 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:14.417504 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0568c-834f-48a5-ba39-6ff968207442 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:14.417995 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0568c-834f-48a5-ba39-6ff968207442 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:14.421589 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4aa0568c-834f-48a5-ba39-6ff968207442 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:14.482321 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:14.507605 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a75eb3-f761-4bcb-b9d9-e3a71319c727 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98560723b49940d38a6a5e9cf3f5d3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:14.508632 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a75eb3-f761-4bcb-b9d9-e3a71319c727 service glance] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:14.509105 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a75eb3-f761-4bcb-b9d9-e3a71319c727 service glance] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:14.509546 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a75eb3-f761-4bcb-b9d9-e3a71319c727 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:14.512769 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-44a75eb3-f761-4bcb-b9d9-e3a71319c727 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:14.519386 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:14.579749 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa0568c-834f-48a5-ba39-6ff968207442 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:14.582551 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa0568c-834f-48a5-ba39-6ff968207442 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:14.616310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 708/1395] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:16:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:16:14.704915 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a75eb3-f761-4bcb-b9d9-e3a71319c727 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:14.707049 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a75eb3-f761-4bcb-b9d9-e3a71319c727 service glance] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:14.737941 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 688/1396] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:16:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:16:15.747326 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc18f8a-b0c6-41c4-beac-eb80813ab1a9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:15.748066 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc18f8a-b0c6-41c4-beac-eb80813ab1a9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:15.748327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc18f8a-b0c6-41c4-beac-eb80813ab1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:15.850620 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1cc18f8a-b0c6-41c4-beac-eb80813ab1a9 None None] MFA Rules not processed for user `1c0772205ae24713b8901666a87018b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:15.852056 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1cc18f8a-b0c6-41c4-beac-eb80813ab1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:15.982876 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 709/1397] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Jul 25 19:16:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:16.036546 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a5623-a817-45a6-87f8-ae5910386910 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:16.037402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a5623-a817-45a6-87f8-ae5910386910 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:16.038081 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a5623-a817-45a6-87f8-ae5910386910 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:16.041700 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 689/1398] 10.222.0.18 () {54 vars in 943 bytes} [Mon Jul 25 19:16:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:16:16.051135 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec63cc7-78c8-4a9f-866a-5b191f9662b9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:16.051764 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec63cc7-78c8-4a9f-866a-5b191f9662b9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:16.052339 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec63cc7-78c8-4a9f-866a-5b191f9662b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:16.111068 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:16.187933 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2ec63cc7-78c8-4a9f-866a-5b191f9662b9 None None] MFA Rules not processed for user `feafcf1b5a9844fe9738823fddd07e0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:16.189577 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2ec63cc7-78c8-4a9f-866a-5b191f9662b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:16.343634 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 710/1399] 10.222.0.18 () {58 vars in 1040 bytes} [Mon Jul 25 19:16:16 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:16.354197 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a5623-a817-45a6-87f8-ae5910386910 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:16.355875 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-566a5623-a817-45a6-87f8-ae5910386910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:16.545223 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:16.613741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a5623-a817-45a6-87f8-ae5910386910 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:16.821843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6a2ea0-6958-48d7-8d81-69557cc5aab3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feafcf1b5a9844fe9738823fddd07e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:16.822920 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a2ea0-6958-48d7-8d81-69557cc5aab3 service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:16.823404 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a2ea0-6958-48d7-8d81-69557cc5aab3 service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:16.823873 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a2ea0-6958-48d7-8d81-69557cc5aab3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:16.827402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6a6a2ea0-6958-48d7-8d81-69557cc5aab3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:16.948841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6a2ea0-6958-48d7-8d81-69557cc5aab3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:16.951373 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6a2ea0-6958-48d7-8d81-69557cc5aab3 service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:16.961473 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 690/1400] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Jul 25 19:16:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 19:16:17.159714 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367e875-69d2-4d5d-b13f-c0025e7bfeb0 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:17.160567 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367e875-69d2-4d5d-b13f-c0025e7bfeb0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:17.160567 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367e875-69d2-4d5d-b13f-c0025e7bfeb0 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:17.161744 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 711/1401] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:16:17.168305 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5320bdd-01fa-4649-91fc-4d83d4a2dc53 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:17.169639 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5320bdd-01fa-4649-91fc-4d83d4a2dc53 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:17.169639 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5320bdd-01fa-4649-91fc-4d83d4a2dc53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:17.181089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:17.282324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f5320bdd-01fa-4649-91fc-4d83d4a2dc53 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:17.284928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f5320bdd-01fa-4649-91fc-4d83d4a2dc53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:17.509079 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 691/1402] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:16:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:17.572392 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6367e875-69d2-4d5d-b13f-c0025e7bfeb0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:17.574113 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6367e875-69d2-4d5d-b13f-c0025e7bfeb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:17.764144 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:17.927138 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6367e875-69d2-4d5d-b13f-c0025e7bfeb0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:18.120388 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2fafdd-76d1-438d-9fd4-05b208c2a864 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:18.121057 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2fafdd-76d1-438d-9fd4-05b208c2a864 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:18.121375 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2fafdd-76d1-438d-9fd4-05b208c2a864 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:18.121765 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2fafdd-76d1-438d-9fd4-05b208c2a864 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:18.123744 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-de2fafdd-76d1-438d-9fd4-05b208c2a864 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:18.191902 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:18.256904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2fafdd-76d1-438d-9fd4-05b208c2a864 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:18.258645 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2fafdd-76d1-438d-9fd4-05b208c2a864 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:18.285672 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 712/1403] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:16:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 713 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:16:21.253098 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf33bb-a10a-4c72-b0f3-04de8bdfff1b None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:21.254023 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf33bb-a10a-4c72-b0f3-04de8bdfff1b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:21.254185 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf33bb-a10a-4c72-b0f3-04de8bdfff1b None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:21.265780 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 692/1404] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:16:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:16:21.266590 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d78cc2-2781-4be9-b3ba-011dbcbe185d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:21.267805 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d78cc2-2781-4be9-b3ba-011dbcbe185d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:21.267805 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d78cc2-2781-4be9-b3ba-011dbcbe185d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:21.418060 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-08d78cc2-2781-4be9-b3ba-011dbcbe185d None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:21.421859 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-08d78cc2-2781-4be9-b3ba-011dbcbe185d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:21.661037 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 713/1405] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:16:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:16:21.748855 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf33bb-a10a-4c72-b0f3-04de8bdfff1b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:21.750979 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fdaf33bb-a10a-4c72-b0f3-04de8bdfff1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:21.947976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:22.082419 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf33bb-a10a-4c72-b0f3-04de8bdfff1b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:22.313893 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d02ef-e2bf-4b5b-9d04-68339ffaf9ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:22.314682 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d02ef-e2bf-4b5b-9d04-68339ffaf9ee service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:22.315037 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d02ef-e2bf-4b5b-9d04-68339ffaf9ee service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:22.315324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d02ef-e2bf-4b5b-9d04-68339ffaf9ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:22.317901 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-530d02ef-e2bf-4b5b-9d04-68339ffaf9ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:22.401986 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:22.501040 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d02ef-e2bf-4b5b-9d04-68339ffaf9ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:22.503569 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d02ef-e2bf-4b5b-9d04-68339ffaf9ee service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:22.536518 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 693/1406] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:16:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 790 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:16:25.710517 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ba01b-1266-4141-982f-0b6ed22b1013 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:25.711334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ba01b-1266-4141-982f-0b6ed22b1013 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:25.711334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ba01b-1266-4141-982f-0b6ed22b1013 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:25.712933 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 714/1407] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:16:25.721529 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d6941-959b-44d3-90c0-667d0288fcab None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:25.722926 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d6941-959b-44d3-90c0-667d0288fcab None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:25.723448 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d6941-959b-44d3-90c0-667d0288fcab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:25.867620 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-218d6941-959b-44d3-90c0-667d0288fcab None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:25.869194 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-218d6941-959b-44d3-90c0-667d0288fcab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:26.039191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 694/1408] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:16:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:26.089768 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ba01b-1266-4141-982f-0b6ed22b1013 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:26.093065 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b03ba01b-1266-4141-982f-0b6ed22b1013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:26.135017 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:26.431318 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ba01b-1266-4141-982f-0b6ed22b1013 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:26.674660 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:26.724961 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b274a3a0-4db5-44c2-87c8-7507d4d34a42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:26.725655 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274a3a0-4db5-44c2-87c8-7507d4d34a42 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:26.725991 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274a3a0-4db5-44c2-87c8-7507d4d34a42 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:26.726344 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274a3a0-4db5-44c2-87c8-7507d4d34a42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:26.729446 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b274a3a0-4db5-44c2-87c8-7507d4d34a42 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:26.891002 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b274a3a0-4db5-44c2-87c8-7507d4d34a42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:26.892905 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b274a3a0-4db5-44c2-87c8-7507d4d34a42 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:26.930628 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 715/1409] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:16:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 842 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:16:36.030888 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:36.924183 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=54 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:44.877771 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3f19a-9458-41b0-bf31-0e6d9e14b612 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:44.878409 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3f19a-9458-41b0-bf31-0e6d9e14b612 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:44.878409 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3f19a-9458-41b0-bf31-0e6d9e14b612 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:44.879981 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 695/1410] 10.222.0.18 () {54 vars in 918 bytes} [Mon Jul 25 19:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:16:44.886728 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414227e-b84f-4fee-8b27-2b7b36b60820 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:44.887536 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414227e-b84f-4fee-8b27-2b7b36b60820 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:44.887536 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414227e-b84f-4fee-8b27-2b7b36b60820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:44.985002 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b414227e-b84f-4fee-8b27-2b7b36b60820 None None] MFA Rules not processed for user `0422d6695bf14925bf0f993cb16e64ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:44.987081 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b414227e-b84f-4fee-8b27-2b7b36b60820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:45.139809 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 716/1411] 10.222.0.18 () {58 vars in 1015 bytes} [Mon Jul 25 19:16:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:16:45.163134 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46579761-253d-4360-b237-bd06e697aeb7 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:45.163630 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46579761-253d-4360-b237-bd06e697aeb7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:45.164034 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46579761-253d-4360-b237-bd06e697aeb7 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:45.165832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 696/1412] 10.222.0.18 () {54 vars in 918 bytes} [Mon Jul 25 19:16:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:16:45.172306 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362c355-2103-42bd-a8b1-fcb19f5e69e2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:45.172676 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362c355-2103-42bd-a8b1-fcb19f5e69e2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:45.173945 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362c355-2103-42bd-a8b1-fcb19f5e69e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:45.274173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:45.279185 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7362c355-2103-42bd-a8b1-fcb19f5e69e2 None None] MFA Rules not processed for user `860e7d26ad364795bb25836aadd4ac7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:45.280969 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7362c355-2103-42bd-a8b1-fcb19f5e69e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:45.437059 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 717/1413] 10.222.0.18 () {58 vars in 1015 bytes} [Mon Jul 25 19:16:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:48.184549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f6d0d-b0a7-4097-974c-ec850e8c7c9d None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:48.185158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f6d0d-b0a7-4097-974c-ec850e8c7c9d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:48.185355 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f6d0d-b0a7-4097-974c-ec850e8c7c9d None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:48.187105 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 697/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 19:16:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:16:48.194310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ac8cd-5a1c-4071-b7a3-2935357a44e1 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:48.195226 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ac8cd-5a1c-4071-b7a3-2935357a44e1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:48.195226 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ac8cd-5a1c-4071-b7a3-2935357a44e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:48.290343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f93ac8cd-5a1c-4071-b7a3-2935357a44e1 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:48.293101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f93ac8cd-5a1c-4071-b7a3-2935357a44e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:48.399367 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:48.455474 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 718/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 19:16:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:16:48.503825 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f6d0d-b0a7-4097-974c-ec850e8c7c9d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:48.506438 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8e6f6d0d-b0a7-4097-974c-ec850e8c7c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:48.705444 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f6d0d-b0a7-4097-974c-ec850e8c7c9d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:48.802029 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:48.865411 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d2d98-1b30-4961-97b0-ecb010a7c8ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:48.866468 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d2d98-1b30-4961-97b0-ecb010a7c8ff service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:48.866949 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d2d98-1b30-4961-97b0-ecb010a7c8ff service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:48.867424 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d2d98-1b30-4961-97b0-ecb010a7c8ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:48.871884 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-296d2d98-1b30-4961-97b0-ecb010a7c8ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:49.010840 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d2d98-1b30-4961-97b0-ecb010a7c8ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:49.014593 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d2d98-1b30-4961-97b0-ecb010a7c8ff service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:49.039318 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 698/1416] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:16:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:16:50.131370 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194be54-6774-4361-a8d0-6f74175f6a84 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:50.132065 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194be54-6774-4361-a8d0-6f74175f6a84 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:50.132065 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194be54-6774-4361-a8d0-6f74175f6a84 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:50.133595 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 719/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 19:16:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:16:50.140096 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0cda52-09e9-4e50-96a7-fd84a8aaae0b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:50.140965 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0cda52-09e9-4e50-96a7-fd84a8aaae0b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:50.141150 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0cda52-09e9-4e50-96a7-fd84a8aaae0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:50.250386 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-bc0cda52-09e9-4e50-96a7-fd84a8aaae0b None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:50.251769 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bc0cda52-09e9-4e50-96a7-fd84a8aaae0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:50.285318 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:50.380597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 699/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 19:16:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:16:50.540388 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194be54-6774-4361-a8d0-6f74175f6a84 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:50.542993 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b194be54-6774-4361-a8d0-6f74175f6a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:50.733289 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194be54-6774-4361-a8d0-6f74175f6a84 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:50.787283 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:50.884869 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f2ea12-91c4-415d-8201-5ac26e3c69a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:50.885684 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2ea12-91c4-415d-8201-5ac26e3c69a1 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:50.886060 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2ea12-91c4-415d-8201-5ac26e3c69a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:50.887754 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2ea12-91c4-415d-8201-5ac26e3c69a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:50.889924 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a1f2ea12-91c4-415d-8201-5ac26e3c69a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:51.036310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f2ea12-91c4-415d-8201-5ac26e3c69a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:51.038046 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f2ea12-91c4-415d-8201-5ac26e3c69a1 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:51.061669 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 720/1419] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:16:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:16:52.133297 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657206ed-629a-4800-b964-d9e8738225ef None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:52.133885 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657206ed-629a-4800-b964-d9e8738225ef None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:52.133885 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657206ed-629a-4800-b964-d9e8738225ef None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:52.135322 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 700/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 19:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:16:52.142094 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aedb800-232d-4c0c-a45d-5f3498370e7a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:52.142962 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aedb800-232d-4c0c-a45d-5f3498370e7a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:52.143125 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aedb800-232d-4c0c-a45d-5f3498370e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:52.213834 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:52.246810 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9aedb800-232d-4c0c-a45d-5f3498370e7a None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:52.248585 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9aedb800-232d-4c0c-a45d-5f3498370e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:52.422457 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 721/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 19:16:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:52.588025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657206ed-629a-4800-b964-d9e8738225ef None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:52.591016 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-657206ed-629a-4800-b964-d9e8738225ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:52.783025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657206ed-629a-4800-b964-d9e8738225ef None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:52.802613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:52.959409 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeba1b53-797b-42e4-a4a6-6b3f94ddd9c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:52.960136 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba1b53-797b-42e4-a4a6-6b3f94ddd9c7 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:52.960606 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba1b53-797b-42e4-a4a6-6b3f94ddd9c7 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:52.960897 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba1b53-797b-42e4-a4a6-6b3f94ddd9c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:52.962721 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-aeba1b53-797b-42e4-a4a6-6b3f94ddd9c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:53.098832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeba1b53-797b-42e4-a4a6-6b3f94ddd9c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:53.102372 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeba1b53-797b-42e4-a4a6-6b3f94ddd9c7 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:53.143181 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 701/1422] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:16:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:16:54.239380 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0082097-71cf-471d-baa8-dd54bb591066 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:54.240116 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0082097-71cf-471d-baa8-dd54bb591066 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:54.240116 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0082097-71cf-471d-baa8-dd54bb591066 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:54.241540 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 722/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 19:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:16:54.247906 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabea3d-811b-4b20-ba9f-77be27ebfab5 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:54.248458 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabea3d-811b-4b20-ba9f-77be27ebfab5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:54.248553 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabea3d-811b-4b20-ba9f-77be27ebfab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:54.272731 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:54.354984 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4cabea3d-811b-4b20-ba9f-77be27ebfab5 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:54.356589 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4cabea3d-811b-4b20-ba9f-77be27ebfab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:54.483122 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 702/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 19:16:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:54.655228 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0082097-71cf-471d-baa8-dd54bb591066 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:54.657187 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e0082097-71cf-471d-baa8-dd54bb591066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:54.791773 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:54.845655 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0082097-71cf-471d-baa8-dd54bb591066 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:54.992292 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb64a3-4e4c-43d6-9239-9cff2b8a3b2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:54.993066 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb64a3-4e4c-43d6-9239-9cff2b8a3b2a service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:54.993435 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb64a3-4e4c-43d6-9239-9cff2b8a3b2a service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:54.993765 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb64a3-4e4c-43d6-9239-9cff2b8a3b2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:54.995948 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e9fb64a3-4e4c-43d6-9239-9cff2b8a3b2a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:55.255296 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:55.275656 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fb64a3-4e4c-43d6-9239-9cff2b8a3b2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:55.277903 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fb64a3-4e4c-43d6-9239-9cff2b8a3b2a service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:55.303531 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 723/1425] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:16:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 649 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:16:56.265538 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf61fd-91a6-46a6-a19e-576dbe21cb0b None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:56.266365 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf61fd-91a6-46a6-a19e-576dbe21cb0b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:56.266529 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf61fd-91a6-46a6-a19e-576dbe21cb0b None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:56.268174 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 703/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 19:16:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:16:56.274732 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c30c3a-cef4-4f74-8d34-cbd4787a323a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:56.275373 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c30c3a-cef4-4f74-8d34-cbd4787a323a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:56.275373 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c30c3a-cef4-4f74-8d34-cbd4787a323a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:56.398562 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e4c30c3a-cef4-4f74-8d34-cbd4787a323a None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:56.400712 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e4c30c3a-cef4-4f74-8d34-cbd4787a323a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:56.530180 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 724/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 19:16:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:56.698107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf61fd-91a6-46a6-a19e-576dbe21cb0b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:56.701606 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-dcaf61fd-91a6-46a6-a19e-576dbe21cb0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:56.815728 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:56.894684 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf61fd-91a6-46a6-a19e-576dbe21cb0b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:57.054607 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aea4ab-7aef-45ef-a513-9878fbfc0a1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:57.055281 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aea4ab-7aef-45ef-a513-9878fbfc0a1c service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:57.055597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aea4ab-7aef-45ef-a513-9878fbfc0a1c service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:57.055881 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aea4ab-7aef-45ef-a513-9878fbfc0a1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:57.057664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-78aea4ab-7aef-45ef-a513-9878fbfc0a1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:57.118885 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:57.171935 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aea4ab-7aef-45ef-a513-9878fbfc0a1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:57.173774 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aea4ab-7aef-45ef-a513-9878fbfc0a1c service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:57.195562 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 704/1428] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:16:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:16:58.187924 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6f0ec-6438-4124-925c-55afe150d48c None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:58.188873 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6f0ec-6438-4124-925c-55afe150d48c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:58.188873 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6f0ec-6438-4124-925c-55afe150d48c None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:58.190151 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 725/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 19:16:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:16:58.196001 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a727ce-2765-479d-8319-bbfce30dc958 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:58.196904 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a727ce-2765-479d-8319-bbfce30dc958 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:58.197083 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a727ce-2765-479d-8319-bbfce30dc958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:58.296746 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-27a727ce-2765-479d-8319-bbfce30dc958 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:16:58.298238 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-27a727ce-2765-479d-8319-bbfce30dc958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:58.427410 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 705/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 19:16:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:16:58.589412 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e6f0ec-6438-4124-925c-55afe150d48c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:16:58.592606 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-39e6f0ec-6438-4124-925c-55afe150d48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:58.687116 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:58.835828 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e6f0ec-6438-4124-925c-55afe150d48c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:16:58.989527 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d8799-fb55-4174-bb38-cdbe3cdc9f73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:16:58.990078 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d8799-fb55-4174-bb38-cdbe3cdc9f73 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:16:58.990423 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d8799-fb55-4174-bb38-cdbe3cdc9f73 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:16:58.990635 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d8799-fb55-4174-bb38-cdbe3cdc9f73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:16:58.992409 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-504d8799-fb55-4174-bb38-cdbe3cdc9f73 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:16:59.001299 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:16:59.122097 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504d8799-fb55-4174-bb38-cdbe3cdc9f73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:16:59.125215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504d8799-fb55-4174-bb38-cdbe3cdc9f73 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:16:59.161714 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 726/1431] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:16:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:17:00.082315 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24628186-310d-4026-8412-f75de367ccb6 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:00.082936 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24628186-310d-4026-8412-f75de367ccb6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:00.082936 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24628186-310d-4026-8412-f75de367ccb6 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:00.084311 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 706/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 19:17:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:17:00.091257 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ba542-6287-4973-ba1f-5f7ac250c1d0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:00.091897 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ba542-6287-4973-ba1f-5f7ac250c1d0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:00.091897 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ba542-6287-4973-ba1f-5f7ac250c1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:00.179887 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5c7ba542-6287-4973-ba1f-5f7ac250c1d0 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:17:00.181150 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5c7ba542-6287-4973-ba1f-5f7ac250c1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:00.260436 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:17:00.298708 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 727/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 19:17:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:17:00.480193 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24628186-310d-4026-8412-f75de367ccb6 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:17:00.481969 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-24628186-310d-4026-8412-f75de367ccb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:00.503062 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:17:00.670752 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24628186-310d-4026-8412-f75de367ccb6 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:17:00.789479 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:17:00.844793 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca0fa8-8d60-4911-b773-db0c577dae0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:17:00.845940 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca0fa8-8d60-4911-b773-db0c577dae0f service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:00.846444 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca0fa8-8d60-4911-b773-db0c577dae0f service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:00.846918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca0fa8-8d60-4911-b773-db0c577dae0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:00.850325 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ffca0fa8-8d60-4911-b773-db0c577dae0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:00.982008 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca0fa8-8d60-4911-b773-db0c577dae0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:17:00.983693 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca0fa8-8d60-4911-b773-db0c577dae0f service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:17:01.006958 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 707/1434] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:17:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:17:01.934825 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd75fed-dc1a-4735-a018-aa715c82a733 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:01.935738 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd75fed-dc1a-4735-a018-aa715c82a733 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:01.935738 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd75fed-dc1a-4735-a018-aa715c82a733 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:01.937757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 728/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 19:17:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:17:01.944084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170cbcc-26d5-43cb-bb57-86c1780f5b6a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:01.944937 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170cbcc-26d5-43cb-bb57-86c1780f5b6a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:01.944937 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170cbcc-26d5-43cb-bb57-86c1780f5b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:02.064578 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5170cbcc-26d5-43cb-bb57-86c1780f5b6a None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:17:02.065903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5170cbcc-26d5-43cb-bb57-86c1780f5b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:02.117617 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:17:02.207003 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 708/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 19:17:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:17:02.379382 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd75fed-dc1a-4735-a018-aa715c82a733 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:17:02.381007 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1cd75fed-dc1a-4735-a018-aa715c82a733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:02.566287 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd75fed-dc1a-4735-a018-aa715c82a733 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:17:02.637171 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:17:02.713443 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a9c1b-71a5-42d2-b340-37bc3a18c12d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:17:02.714716 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a9c1b-71a5-42d2-b340-37bc3a18c12d service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:02.715014 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a9c1b-71a5-42d2-b340-37bc3a18c12d service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:02.715561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a9c1b-71a5-42d2-b340-37bc3a18c12d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:02.717838 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-eb8a9c1b-71a5-42d2-b340-37bc3a18c12d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:02.840869 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8a9c1b-71a5-42d2-b340-37bc3a18c12d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:17:02.842613 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8a9c1b-71a5-42d2-b340-37bc3a18c12d service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:17:02.865293 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 729/1437] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:17:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:17:03.774153 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3a0ce-0701-48cc-a690-5c31231c57e7 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:03.774809 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3a0ce-0701-48cc-a690-5c31231c57e7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:03.774809 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3a0ce-0701-48cc-a690-5c31231c57e7 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:03.776166 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 709/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 19:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:17:03.783102 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17c992-0522-4048-a224-802356e01cfc None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:03.783740 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17c992-0522-4048-a224-802356e01cfc None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:03.783740 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17c992-0522-4048-a224-802356e01cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:03.883954 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6e17c992-0522-4048-a224-802356e01cfc None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:17:03.885237 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6e17c992-0522-4048-a224-802356e01cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:03.887964 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:17:04.044506 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 730/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 19:17:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:17:04.237349 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3a0ce-0701-48cc-a690-5c31231c57e7 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:17:04.239257 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a9c3a0ce-0701-48cc-a690-5c31231c57e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:04.450627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3a0ce-0701-48cc-a690-5c31231c57e7 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:17:04.487930 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:17:04.614265 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7f787-d5f3-4689-8f8f-dd3f7dead3e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:17:04.614938 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7f787-d5f3-4689-8f8f-dd3f7dead3e1 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:04.615275 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7f787-d5f3-4689-8f8f-dd3f7dead3e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:04.615576 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7f787-d5f3-4689-8f8f-dd3f7dead3e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:04.617484 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-58d7f787-d5f3-4689-8f8f-dd3f7dead3e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:04.766132 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d7f787-d5f3-4689-8f8f-dd3f7dead3e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:17:04.767960 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d7f787-d5f3-4689-8f8f-dd3f7dead3e1 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:17:04.798655 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 710/1440] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:17:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:17:06.940976 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9b771-f58b-4a7a-81a8-31e1b7a1e73b None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:06.941819 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9b771-f58b-4a7a-81a8-31e1b7a1e73b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:06.941819 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9b771-f58b-4a7a-81a8-31e1b7a1e73b None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:06.943186 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 731/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 19:17:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:17:06.951557 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f4922-0938-40cc-8b85-15516b9f897d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:06.952246 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f4922-0938-40cc-8b85-15516b9f897d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:06.952395 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f4922-0938-40cc-8b85-15516b9f897d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:07.006293 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:17:07.046233 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ef1f4922-0938-40cc-8b85-15516b9f897d None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:17:07.048453 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ef1f4922-0938-40cc-8b85-15516b9f897d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:07.215791 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 711/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 19:17:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:17:07.261305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9b771-f58b-4a7a-81a8-31e1b7a1e73b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:17:07.263313 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-19b9b771-f58b-4a7a-81a8-31e1b7a1e73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:07.442084 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9b771-f58b-4a7a-81a8-31e1b7a1e73b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:17:07.506035 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:17:07.582216 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53860585-fc96-4bc0-8d25-7d979f5d4cc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:17:07.582929 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53860585-fc96-4bc0-8d25-7d979f5d4cc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:07.583216 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53860585-fc96-4bc0-8d25-7d979f5d4cc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:07.583497 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53860585-fc96-4bc0-8d25-7d979f5d4cc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:07.585306 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-53860585-fc96-4bc0-8d25-7d979f5d4cc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:07.784133 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53860585-fc96-4bc0-8d25-7d979f5d4cc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:17:07.785515 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53860585-fc96-4bc0-8d25-7d979f5d4cc7 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:17:07.815145 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 732/1443] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:17:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:17:10.909850 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b35b67-c115-422e-9c8c-c99cf951a183 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:10.910779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b35b67-c115-422e-9c8c-c99cf951a183 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:10.910779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b35b67-c115-422e-9c8c-c99cf951a183 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:10.912181 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 712/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 19:17:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:17:10.923095 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a86adf-9219-4fc7-acef-7cb2e6860308 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:10.923717 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a86adf-9219-4fc7-acef-7cb2e6860308 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:10.924105 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a86adf-9219-4fc7-acef-7cb2e6860308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:11.027900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-10a86adf-9219-4fc7-acef-7cb2e6860308 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:17:11.029190 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-10a86adf-9219-4fc7-acef-7cb2e6860308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:11.049517 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:17:11.180881 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 733/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 19:17:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:17:11.251794 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b35b67-c115-422e-9c8c-c99cf951a183 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:17:11.253498 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-86b35b67-c115-422e-9c8c-c99cf951a183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:11.446510 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b35b67-c115-422e-9c8c-c99cf951a183 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:17:11.471348 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:17:11.572322 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ac8644-2d1b-411d-a915-b8a24ad734db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:17:11.574470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac8644-2d1b-411d-a915-b8a24ad734db service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:11.574819 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac8644-2d1b-411d-a915-b8a24ad734db service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:11.575150 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac8644-2d1b-411d-a915-b8a24ad734db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:11.577287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-66ac8644-2d1b-411d-a915-b8a24ad734db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:11.749582 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ac8644-2d1b-411d-a915-b8a24ad734db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:17:11.750965 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ac8644-2d1b-411d-a915-b8a24ad734db service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:17:11.774201 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 713/1446] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:17:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:17:14.714819 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886028b-8e0d-4dc4-84d6-18446d95ff2f None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:14.715911 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886028b-8e0d-4dc4-84d6-18446d95ff2f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:14.716173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886028b-8e0d-4dc4-84d6-18446d95ff2f None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:14.719025 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 734/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 19:17:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:17:14.728819 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb903c-5bf4-453c-bf0e-817f0b100e4b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:14.730067 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb903c-5bf4-453c-bf0e-817f0b100e4b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:14.730067 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb903c-5bf4-453c-bf0e-817f0b100e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:14.790824 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:17:14.815706 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c8bb903c-5bf4-453c-bf0e-817f0b100e4b None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:17:14.817153 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c8bb903c-5bf4-453c-bf0e-817f0b100e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:14.947438 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 714/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 19:17:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:17:14.996397 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886028b-8e0d-4dc4-84d6-18446d95ff2f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:17:15.004284 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2886028b-8e0d-4dc4-84d6-18446d95ff2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:15.144920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886028b-8e0d-4dc4-84d6-18446d95ff2f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:17:15.199119 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:17:15.250208 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c510de-86e7-4972-9539-1bbe6ffe6ed6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:17:15.250749 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c510de-86e7-4972-9539-1bbe6ffe6ed6 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:15.251038 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c510de-86e7-4972-9539-1bbe6ffe6ed6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:15.251412 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c510de-86e7-4972-9539-1bbe6ffe6ed6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:15.253658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e2c510de-86e7-4972-9539-1bbe6ffe6ed6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:15.418265 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c510de-86e7-4972-9539-1bbe6ffe6ed6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:17:15.420278 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c510de-86e7-4972-9539-1bbe6ffe6ed6 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:17:15.451368 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 735/1449] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:17:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:17:17.998852 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea8559-3c0d-409a-bd41-70a2a7815ee4 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:17.999565 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea8559-3c0d-409a-bd41-70a2a7815ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:17.999902 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea8559-3c0d-409a-bd41-70a2a7815ee4 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:18.002619 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 715/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 19:17:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:17:18.012155 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265858e-0b82-4b8a-aad2-f1fa8dbba310 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:18.013203 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265858e-0b82-4b8a-aad2-f1fa8dbba310 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:18.013203 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265858e-0b82-4b8a-aad2-f1fa8dbba310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:18.097077 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6265858e-0b82-4b8a-aad2-f1fa8dbba310 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:17:18.098551 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6265858e-0b82-4b8a-aad2-f1fa8dbba310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:18.124858 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:17:18.226964 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 736/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 19:17:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:17:18.269392 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea8559-3c0d-409a-bd41-70a2a7815ee4 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:17:18.272474 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-06ea8559-3c0d-409a-bd41-70a2a7815ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:18.430718 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea8559-3c0d-409a-bd41-70a2a7815ee4 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:17:18.458812 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:17:18.548033 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1eb9f4-9870-4af6-a06a-5876a0727631 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:17:18.548589 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1eb9f4-9870-4af6-a06a-5876a0727631 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:18.548873 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1eb9f4-9870-4af6-a06a-5876a0727631 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:18.549146 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1eb9f4-9870-4af6-a06a-5876a0727631 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:18.551288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ce1eb9f4-9870-4af6-a06a-5876a0727631 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:18.715745 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1eb9f4-9870-4af6-a06a-5876a0727631 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:17:18.718728 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1eb9f4-9870-4af6-a06a-5876a0727631 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:17:18.764771 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 716/1452] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:17:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:17:21.149285 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85407f-f52f-4d21-b867-b05847c0ccaa None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:21.149972 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85407f-f52f-4d21-b867-b05847c0ccaa None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:21.149972 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85407f-f52f-4d21-b867-b05847c0ccaa None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:21.152614 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 737/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 19:17:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:17:21.158995 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e7502-ae02-415a-aa74-858e84dd66d2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:21.159637 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e7502-ae02-415a-aa74-858e84dd66d2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:21.159637 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e7502-ae02-415a-aa74-858e84dd66d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:21.264512 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:17:21.269525 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1c7e7502-ae02-415a-aa74-858e84dd66d2 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:17:21.271849 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1c7e7502-ae02-415a-aa74-858e84dd66d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:21.408090 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 717/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 19:17:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:17:21.450166 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85407f-f52f-4d21-b867-b05847c0ccaa None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:17:21.453234 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0a85407f-f52f-4d21-b867-b05847c0ccaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:21.681859 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85407f-f52f-4d21-b867-b05847c0ccaa None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:17:21.745072 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:17:21.777462 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579ecb07-2e82-43cf-bdba-2e0440e0241b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:17:21.778139 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ecb07-2e82-43cf-bdba-2e0440e0241b service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:21.778423 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ecb07-2e82-43cf-bdba-2e0440e0241b service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:21.778703 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579ecb07-2e82-43cf-bdba-2e0440e0241b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:21.780618 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-579ecb07-2e82-43cf-bdba-2e0440e0241b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:21.925471 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579ecb07-2e82-43cf-bdba-2e0440e0241b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:17:21.926849 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579ecb07-2e82-43cf-bdba-2e0440e0241b service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:17:21.949109 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 738/1455] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:17:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:17:24.422227 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d84d6-1743-4d59-b1bd-2db3d9e4d0a1 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:24.428758 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d84d6-1743-4d59-b1bd-2db3d9e4d0a1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:24.429478 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d84d6-1743-4d59-b1bd-2db3d9e4d0a1 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:24.432266 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 718/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 19:17:24 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:17:24.441439 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb68aed-4669-4e83-91d8-b4bc5345822d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:24.442179 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb68aed-4669-4e83-91d8-b4bc5345822d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:24.442311 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb68aed-4669-4e83-91d8-b4bc5345822d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:24.557930 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ddb68aed-4669-4e83-91d8-b4bc5345822d None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:17:24.559728 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ddb68aed-4669-4e83-91d8-b4bc5345822d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:24.612206 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:17:24.707304 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 739/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 19:17:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:17:24.748644 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d84d6-1743-4d59-b1bd-2db3d9e4d0a1 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:17:24.750228 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a93d84d6-1743-4d59-b1bd-2db3d9e4d0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:24.877538 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d84d6-1743-4d59-b1bd-2db3d9e4d0a1 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:17:24.913063 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:17:24.968070 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c05ecb-87c5-4431-8313-4361bc107b6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:17:24.968648 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c05ecb-87c5-4431-8313-4361bc107b6a service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:24.968935 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c05ecb-87c5-4431-8313-4361bc107b6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:24.969213 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c05ecb-87c5-4431-8313-4361bc107b6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:24.971015 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-55c05ecb-87c5-4431-8313-4361bc107b6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:25.137139 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c05ecb-87c5-4431-8313-4361bc107b6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:17:25.138642 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c05ecb-87c5-4431-8313-4361bc107b6a service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:17:25.162432 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 719/1458] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:17:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:17:27.596912 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22955e7b-65d8-449f-93f5-d71d4abf2a83 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:27.597698 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22955e7b-65d8-449f-93f5-d71d4abf2a83 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:27.597698 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22955e7b-65d8-449f-93f5-d71d4abf2a83 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:27.599102 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 740/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 19:17:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:17:27.606230 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d17780f-6956-4684-97de-f3fb65f9164d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:27.606835 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d17780f-6956-4684-97de-f3fb65f9164d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:27.606835 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d17780f-6956-4684-97de-f3fb65f9164d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:27.694976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3d17780f-6956-4684-97de-f3fb65f9164d None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:17:27.696417 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3d17780f-6956-4684-97de-f3fb65f9164d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:27.708493 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:17:27.833643 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 720/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 19:17:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:17:27.878542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22955e7b-65d8-449f-93f5-d71d4abf2a83 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:17:27.880449 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-22955e7b-65d8-449f-93f5-d71d4abf2a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:28.025093 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22955e7b-65d8-449f-93f5-d71d4abf2a83 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:17:28.101294 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:17:28.126794 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea388c81-29d7-4066-b61d-f6b34fd71e95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:17:28.127458 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea388c81-29d7-4066-b61d-f6b34fd71e95 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:28.127845 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea388c81-29d7-4066-b61d-f6b34fd71e95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:28.128168 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea388c81-29d7-4066-b61d-f6b34fd71e95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:28.130058 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ea388c81-29d7-4066-b61d-f6b34fd71e95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:28.310918 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea388c81-29d7-4066-b61d-f6b34fd71e95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:17:28.312676 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea388c81-29d7-4066-b61d-f6b34fd71e95 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:17:28.338076 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 741/1461] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:17:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:17:30.676273 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3cbfe-72be-4a97-a77a-38e8c2e2e381 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:30.677206 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3cbfe-72be-4a97-a77a-38e8c2e2e381 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:30.677605 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3cbfe-72be-4a97-a77a-38e8c2e2e381 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:30.679177 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 721/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 19:17:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:17:30.688125 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b823c51-d8d0-47dc-8e3c-4573cb7dc700 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:30.688754 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b823c51-d8d0-47dc-8e3c-4573cb7dc700 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:30.689127 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b823c51-d8d0-47dc-8e3c-4573cb7dc700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:30.782027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6b823c51-d8d0-47dc-8e3c-4573cb7dc700 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:17:30.783214 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6b823c51-d8d0-47dc-8e3c-4573cb7dc700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:30.837433 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:17:30.923208 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 742/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 19:17:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:17:30.967881 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a3cbfe-72be-4a97-a77a-38e8c2e2e381 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:17:30.970701 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d3a3cbfe-72be-4a97-a77a-38e8c2e2e381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:31.131443 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a3cbfe-72be-4a97-a77a-38e8c2e2e381 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:17:31.188818 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:17:31.241607 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e0520-1d83-41d3-8b4f-089ee5e7ff7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:17:31.242282 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e0520-1d83-41d3-8b4f-089ee5e7ff7d service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:31.242578 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e0520-1d83-41d3-8b4f-089ee5e7ff7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:31.242854 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e0520-1d83-41d3-8b4f-089ee5e7ff7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:31.244787 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-542e0520-1d83-41d3-8b4f-089ee5e7ff7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:31.407071 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542e0520-1d83-41d3-8b4f-089ee5e7ff7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:17:31.408687 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542e0520-1d83-41d3-8b4f-089ee5e7ff7d service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:17:31.440537 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 722/1464] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:17:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:17:33.690508 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff0ada-5143-49aa-bbc8-17773c0111c0 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:33.691293 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff0ada-5143-49aa-bbc8-17773c0111c0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:33.691293 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff0ada-5143-49aa-bbc8-17773c0111c0 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:33.692471 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 743/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 19:17:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:17:33.698514 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb22a86-3fe9-47b0-9e90-fd0f60e24d5e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:33.699117 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb22a86-3fe9-47b0-9e90-fd0f60e24d5e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:33.699117 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb22a86-3fe9-47b0-9e90-fd0f60e24d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:33.792469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-bbb22a86-3fe9-47b0-9e90-fd0f60e24d5e None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:17:33.793823 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bbb22a86-3fe9-47b0-9e90-fd0f60e24d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:33.823994 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:17:33.933104 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 723/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 19:17:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:17:33.973968 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff0ada-5143-49aa-bbc8-17773c0111c0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:17:33.975737 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a3ff0ada-5143-49aa-bbc8-17773c0111c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:34.123692 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff0ada-5143-49aa-bbc8-17773c0111c0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:17:34.209488 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:17:34.220875 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb2667-a0b2-4db2-a02e-ed22eba213fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:17:34.221609 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb2667-a0b2-4db2-a02e-ed22eba213fe service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:34.221939 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb2667-a0b2-4db2-a02e-ed22eba213fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:34.222295 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb2667-a0b2-4db2-a02e-ed22eba213fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:34.224337 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2fdb2667-a0b2-4db2-a02e-ed22eba213fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:34.369527 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdb2667-a0b2-4db2-a02e-ed22eba213fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:17:34.372017 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdb2667-a0b2-4db2-a02e-ed22eba213fe service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:17:34.393736 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 744/1467] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:17:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:17:36.819106 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d293e-0253-4ab6-9996-4fe1bed5b301 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:36.820102 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d293e-0253-4ab6-9996-4fe1bed5b301 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:36.820102 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d293e-0253-4ab6-9996-4fe1bed5b301 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:36.821442 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 724/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 19:17:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:17:36.830247 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c0a46-8b0a-49c0-8beb-6767d7866572 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:36.831129 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c0a46-8b0a-49c0-8beb-6767d7866572 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:36.831129 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c0a46-8b0a-49c0-8beb-6767d7866572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:36.949048 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b50c0a46-8b0a-49c0-8beb-6767d7866572 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:17:36.953351 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b50c0a46-8b0a-49c0-8beb-6767d7866572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:37.063581 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:17:37.134329 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 745/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 19:17:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:17:37.178772 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d293e-0253-4ab6-9996-4fe1bed5b301 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:17:37.180956 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0a7d293e-0253-4ab6-9996-4fe1bed5b301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:37.345402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d293e-0253-4ab6-9996-4fe1bed5b301 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:17:37.435035 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:17:37.493379 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ee152c-ebf5-488c-842d-37be07e058d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:17:37.495032 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee152c-ebf5-488c-842d-37be07e058d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:17:37.495532 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee152c-ebf5-488c-842d-37be07e058d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:17:37.496043 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee152c-ebf5-488c-842d-37be07e058d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:17:37.499456 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-70ee152c-ebf5-488c-842d-37be07e058d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:17:37.674586 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ee152c-ebf5-488c-842d-37be07e058d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:17:37.676391 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ee152c-ebf5-488c-842d-37be07e058d4 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:17:37.700845 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 725/1470] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:17:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:17:47.131715 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=22 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:17:47.697367 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=66 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:18:00.233208 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f445069-c375-421d-ad8f-96472fbc1665 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:18:00.233862 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f445069-c375-421d-ad8f-96472fbc1665 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:18:00.233862 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f445069-c375-421d-ad8f-96472fbc1665 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:18:00.235187 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 746/1471] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:18:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:18:00.241038 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2df6c0-028b-4cc0-a28c-80b7efaa9217 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:18:00.241578 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2df6c0-028b-4cc0-a28c-80b7efaa9217 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:18:00.241678 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2df6c0-028b-4cc0-a28c-80b7efaa9217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:18:00.325322 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-fc2df6c0-028b-4cc0-a28c-80b7efaa9217 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:18:00.326603 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fc2df6c0-028b-4cc0-a28c-80b7efaa9217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:18:00.456900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 726/1472] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:18:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:18:00.553608 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1355e2-8449-4bf8-a5ef-898037a9cdcb None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:18:00.553972 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1355e2-8449-4bf8-a5ef-898037a9cdcb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:18:00.554309 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1355e2-8449-4bf8-a5ef-898037a9cdcb None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:18:00.556101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 747/1473] 10.222.0.18 () {54 vars in 966 bytes} [Mon Jul 25 19:18:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:18:00.566051 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f8a1c-b6bf-4ff1-afb8-08ce84e92e5c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:18:00.566561 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f8a1c-b6bf-4ff1-afb8-08ce84e92e5c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:18:00.567068 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f8a1c-b6bf-4ff1-afb8-08ce84e92e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:18:00.686350 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ec2f8a1c-b6bf-4ff1-afb8-08ce84e92e5c None None] MFA Rules not processed for user `98560723b49940d38a6a5e9cf3f5d3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:18:00.687793 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ec2f8a1c-b6bf-4ff1-afb8-08ce84e92e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:18:00.718048 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:18:00.799772 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 727/1474] 10.222.0.18 () {58 vars in 1063 bytes} [Mon Jul 25 19:18:00 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:18:00.809236 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1355e2-8449-4bf8-a5ef-898037a9cdcb None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:18:00.811250 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5f1355e2-8449-4bf8-a5ef-898037a9cdcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:18:00.951233 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1355e2-8449-4bf8-a5ef-898037a9cdcb None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:18:01.050808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391206dc-a2a7-4308-b847-833e8a51c708 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98560723b49940d38a6a5e9cf3f5d3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:18:01.051373 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391206dc-a2a7-4308-b847-833e8a51c708 service glance] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:18:01.051695 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391206dc-a2a7-4308-b847-833e8a51c708 service glance] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:18:01.051994 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391206dc-a2a7-4308-b847-833e8a51c708 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:18:01.053753 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-391206dc-a2a7-4308-b847-833e8a51c708 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:18:01.122771 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:18:01.198187 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391206dc-a2a7-4308-b847-833e8a51c708 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:18:01.199970 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391206dc-a2a7-4308-b847-833e8a51c708 service glance] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:18:01.223739 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 748/1475] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:18:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:18:03.934815 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397a492-3712-4d4e-a21f-3f7a12c54845 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:18:03.935601 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397a492-3712-4d4e-a21f-3f7a12c54845 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:18:03.935601 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397a492-3712-4d4e-a21f-3f7a12c54845 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:18:03.937230 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 728/1476] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:18:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:18:03.947943 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6bb07-2043-4225-beab-d6be45e946ac None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:18:03.948542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6bb07-2043-4225-beab-d6be45e946ac None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:18:03.948938 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6bb07-2043-4225-beab-d6be45e946ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:18:04.081281 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-76a6bb07-2043-4225-beab-d6be45e946ac None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:18:04.082589 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-76a6bb07-2043-4225-beab-d6be45e946ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:18:04.214533 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 749/1477] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:18:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:18:04.256275 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a397a492-3712-4d4e-a21f-3f7a12c54845 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:18:04.257903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a397a492-3712-4d4e-a21f-3f7a12c54845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:18:04.427202 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a397a492-3712-4d4e-a21f-3f7a12c54845 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:18:04.505711 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:18:04.564971 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b54f2-b6f1-4ce7-aa94-e823af2198d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:18:04.565748 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b54f2-b6f1-4ce7-aa94-e823af2198d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:18:04.566064 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b54f2-b6f1-4ce7-aa94-e823af2198d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:18:04.566342 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b54f2-b6f1-4ce7-aa94-e823af2198d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:18:04.567392 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b54f2-b6f1-4ce7-aa94-e823af2198d3 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:18:04.569358 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: WARNING py.warnings [None req-457b54f2-b6f1-4ce7-aa94-e823af2198d3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 19:18:04.569358 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: warnings.warn(msg) Jul 25 19:18:04.569358 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: Jul 25 19:18:04.570543 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b54f2-b6f1-4ce7-aa94-e823af2198d3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:18:04.595786 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 729/1478] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Jul 25 19:18:04 2022] GET /identity/v3/projects => generated 2553 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:18:06.973445 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728c019-aa7e-4b15-8974-d0c3cc5072fc None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:18:06.974064 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728c019-aa7e-4b15-8974-d0c3cc5072fc None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:18:06.974064 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728c019-aa7e-4b15-8974-d0c3cc5072fc None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:18:06.975931 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 750/1479] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:18:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:18:06.982592 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb673c-7b6f-461c-94be-634e69164c11 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:18:06.983297 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb673c-7b6f-461c-94be-634e69164c11 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:18:06.983297 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb673c-7b6f-461c-94be-634e69164c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:18:07.068533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5dbb673c-7b6f-461c-94be-634e69164c11 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:18:07.069726 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5dbb673c-7b6f-461c-94be-634e69164c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:18:07.198380 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 730/1480] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:18:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:18:07.261610 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728c019-aa7e-4b15-8974-d0c3cc5072fc None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:18:07.264192 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a728c019-aa7e-4b15-8974-d0c3cc5072fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:18:07.276068 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:18:07.482698 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728c019-aa7e-4b15-8974-d0c3cc5072fc None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:18:07.607838 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:18:07.656911 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8363291-1d19-492a-8f4c-dc1444f737ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:18:07.657668 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8363291-1d19-492a-8f4c-dc1444f737ec service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:18:07.657952 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8363291-1d19-492a-8f4c-dc1444f737ec service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:18:07.658227 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8363291-1d19-492a-8f4c-dc1444f737ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:18:07.660292 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d8363291-1d19-492a-8f4c-dc1444f737ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:18:07.781409 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8363291-1d19-492a-8f4c-dc1444f737ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:18:07.783052 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8363291-1d19-492a-8f4c-dc1444f737ec service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:18:07.806454 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 751/1481] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:18:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:18:10.263119 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe28f5-8d46-4687-a366-ce8044b53292 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:18:10.263806 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe28f5-8d46-4687-a366-ce8044b53292 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:18:10.263991 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe28f5-8d46-4687-a366-ce8044b53292 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:18:10.265768 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 731/1482] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:18:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:18:10.272477 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc494a0-2b15-4cfe-aa26-860df3cac51e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:18:10.273211 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc494a0-2b15-4cfe-aa26-860df3cac51e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:18:10.273211 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc494a0-2b15-4cfe-aa26-860df3cac51e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:18:10.359805 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1bc494a0-2b15-4cfe-aa26-860df3cac51e None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:18:10.361247 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1bc494a0-2b15-4cfe-aa26-860df3cac51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:18:10.447102 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:18:10.513091 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 752/1483] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:18:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:18:10.554298 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe28f5-8d46-4687-a366-ce8044b53292 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:18:10.556290 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d5fe28f5-8d46-4687-a366-ce8044b53292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:18:10.586276 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:18:10.692576 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe28f5-8d46-4687-a366-ce8044b53292 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:18:10.794107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b944e3f-88e6-4fa0-92fe-6a9ac6e3eacb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:18:10.795186 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b944e3f-88e6-4fa0-92fe-6a9ac6e3eacb service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:18:10.795748 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b944e3f-88e6-4fa0-92fe-6a9ac6e3eacb service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:18:10.796274 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b944e3f-88e6-4fa0-92fe-6a9ac6e3eacb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:18:10.799778 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4b944e3f-88e6-4fa0-92fe-6a9ac6e3eacb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:18:10.948117 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:18:11.020287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b944e3f-88e6-4fa0-92fe-6a9ac6e3eacb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:18:11.023116 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b944e3f-88e6-4fa0-92fe-6a9ac6e3eacb service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:18:11.060079 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 732/1484] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:18:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:18:13.481370 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8bcd1-deb3-40bf-b1f7-e7a29089dfdd None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:18:13.482081 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8bcd1-deb3-40bf-b1f7-e7a29089dfdd None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:18:13.482365 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8bcd1-deb3-40bf-b1f7-e7a29089dfdd None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:18:13.485009 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 753/1485] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:18:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:18:13.491223 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73e388-c825-4ee6-a926-a7ef120b33d2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:18:13.491992 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73e388-c825-4ee6-a926-a7ef120b33d2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:18:13.491992 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73e388-c825-4ee6-a926-a7ef120b33d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:18:13.578237 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6e73e388-c825-4ee6-a926-a7ef120b33d2 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:18:13.579680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6e73e388-c825-4ee6-a926-a7ef120b33d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:18:13.709476 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 733/1486] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:18:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:18:13.752135 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8bcd1-deb3-40bf-b1f7-e7a29089dfdd None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:18:13.755080 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7bb8bcd1-deb3-40bf-b1f7-e7a29089dfdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:18:13.905792 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8bcd1-deb3-40bf-b1f7-e7a29089dfdd None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:18:14.047959 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:18:14.050737 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dfbcc2-b344-41d3-8099-dcb16e0ca461 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:18:14.051563 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfbcc2-b344-41d3-8099-dcb16e0ca461 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:18:14.051929 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfbcc2-b344-41d3-8099-dcb16e0ca461 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:18:14.052330 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfbcc2-b344-41d3-8099-dcb16e0ca461 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:18:14.054917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-62dfbcc2-b344-41d3-8099-dcb16e0ca461 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:18:14.230025 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dfbcc2-b344-41d3-8099-dcb16e0ca461 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:18:14.231497 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dfbcc2-b344-41d3-8099-dcb16e0ca461 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:18:14.255332 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 754/1487] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:18:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:18:16.704281 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684bc9f-a4fe-4d5d-93ec-bdf3b75e6e80 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:18:16.704947 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684bc9f-a4fe-4d5d-93ec-bdf3b75e6e80 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:18:16.704947 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684bc9f-a4fe-4d5d-93ec-bdf3b75e6e80 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:18:16.706448 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 734/1488] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:18:16.713152 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c8b62-7b49-4603-b798-ec173a928907 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:18:16.713764 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c8b62-7b49-4603-b798-ec173a928907 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:18:16.713764 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c8b62-7b49-4603-b798-ec173a928907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:18:16.802308 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-fd3c8b62-7b49-4603-b798-ec173a928907 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:18:16.803612 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fd3c8b62-7b49-4603-b798-ec173a928907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:18:16.870574 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:18:16.928493 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 755/1489] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:18:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:18:16.977426 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e684bc9f-a4fe-4d5d-93ec-bdf3b75e6e80 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:18:16.979679 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e684bc9f-a4fe-4d5d-93ec-bdf3b75e6e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:18:17.033212 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:18:17.160995 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e684bc9f-a4fe-4d5d-93ec-bdf3b75e6e80 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:18:17.325851 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a9d402-ba44-4c8a-90f9-2c0dddb9caf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:18:17.326699 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a9d402-ba44-4c8a-90f9-2c0dddb9caf2 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:18:17.327162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a9d402-ba44-4c8a-90f9-2c0dddb9caf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:18:17.327677 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a9d402-ba44-4c8a-90f9-2c0dddb9caf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:18:17.331017 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-58a9d402-ba44-4c8a-90f9-2c0dddb9caf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:18:17.481193 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:18:17.544012 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a9d402-ba44-4c8a-90f9-2c0dddb9caf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:18:17.546814 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a9d402-ba44-4c8a-90f9-2c0dddb9caf2 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:18:17.588146 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 735/1490] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:18:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:18:20.104645 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa58d2-cf2a-4f17-acb6-604d5896d4cf None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:18:20.105418 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa58d2-cf2a-4f17-acb6-604d5896d4cf None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:18:20.105418 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa58d2-cf2a-4f17-acb6-604d5896d4cf None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:18:20.106743 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 756/1491] 10.222.0.18 () {54 vars in 926 bytes} [Mon Jul 25 19:18:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:18:20.113872 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94662f4-c3c8-4c9f-ba74-8ac3947ac3ec None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:18:20.114509 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94662f4-c3c8-4c9f-ba74-8ac3947ac3ec None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:18:20.114839 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94662f4-c3c8-4c9f-ba74-8ac3947ac3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:18:20.345767 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a94662f4-c3c8-4c9f-ba74-8ac3947ac3ec None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:18:20.347048 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a94662f4-c3c8-4c9f-ba74-8ac3947ac3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:18:20.485460 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 736/1492] 10.222.0.18 () {58 vars in 1023 bytes} [Mon Jul 25 19:18:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:18:20.528636 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa58d2-cf2a-4f17-acb6-604d5896d4cf None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:18:20.530978 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-25aa58d2-cf2a-4f17-acb6-604d5896d4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:18:20.713294 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa58d2-cf2a-4f17-acb6-604d5896d4cf None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:18:20.835915 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb19af-e5ff-4315-8708-2dbfc015cf89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:18:20.837692 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb19af-e5ff-4315-8708-2dbfc015cf89 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:18:20.838177 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb19af-e5ff-4315-8708-2dbfc015cf89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:18:20.838628 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb19af-e5ff-4315-8708-2dbfc015cf89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:18:20.841819 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6ceb19af-e5ff-4315-8708-2dbfc015cf89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:18:20.858188 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:18:21.013203 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb19af-e5ff-4315-8708-2dbfc015cf89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:18:21.015122 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb19af-e5ff-4315-8708-2dbfc015cf89 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:18:21.045175 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 757/1493] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:18:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:18:30.482198 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=82 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:18:31.042064 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=50 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:20:05.118445 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b2c16e-b657-45c5-99b7-05787a977da5 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:05.119376 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b2c16e-b657-45c5-99b7-05787a977da5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:05.119376 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b2c16e-b657-45c5-99b7-05787a977da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:05.217937 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-98b2c16e-b657-45c5-99b7-05787a977da5 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:20:05.219277 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-98b2c16e-b657-45c5-99b7-05787a977da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:05.347401 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 737/1494] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:20:05.356350 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f237b-e1bd-4b2b-b928-1d3c7d4f0699 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:05.357062 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f237b-e1bd-4b2b-b928-1d3c7d4f0699 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:05.357062 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f237b-e1bd-4b2b-b928-1d3c7d4f0699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:05.381420 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ba2f237b-e1bd-4b2b-b928-1d3c7d4f0699 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:20:05.382674 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ba2f237b-e1bd-4b2b-b928-1d3c7d4f0699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:05.391944 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 758/1495] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:20:05.397501 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b2c16e-b657-45c5-99b7-05787a977da5 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:05.409198 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b2c16e-b657-45c5-99b7-05787a977da5 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:05.414892 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c19473-10d4-4197-94dc-b30a18598585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:05.415689 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c19473-10d4-4197-94dc-b30a18598585 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:05.415991 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c19473-10d4-4197-94dc-b30a18598585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:05.416285 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c19473-10d4-4197-94dc-b30a18598585 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:05.417044 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c19473-10d4-4197-94dc-b30a18598585 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:20:05.417347 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c19473-10d4-4197-94dc-b30a18598585 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:05.418477 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: WARNING py.warnings [None req-14c19473-10d4-4197-94dc-b30a18598585 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 19:20:05.418477 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: warnings.warn(msg) Jul 25 19:20:05.418477 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: Jul 25 19:20:05.419435 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c19473-10d4-4197-94dc-b30a18598585 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:05.435974 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 738/1496] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:20:05.445852 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2f237b-e1bd-4b2b-b928-1d3c7d4f0699 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:05.464131 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2f237b-e1bd-4b2b-b928-1d3c7d4f0699 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:05.472423 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67fbdaa-e6c0-4b41-b1e9-5895d12867a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:05.475089 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67fbdaa-e6c0-4b41-b1e9-5895d12867a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:05.475650 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67fbdaa-e6c0-4b41-b1e9-5895d12867a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:05.476170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67fbdaa-e6c0-4b41-b1e9-5895d12867a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:05.477517 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67fbdaa-e6c0-4b41-b1e9-5895d12867a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:05.480853 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-a67fbdaa-e6c0-4b41-b1e9-5895d12867a3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 19:20:05.480853 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: warnings.warn(msg) Jul 25 19:20:05.480853 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: Jul 25 19:20:05.482849 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67fbdaa-e6c0-4b41-b1e9-5895d12867a3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:05.535094 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 759/1497] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:20:05 2022] POST /identity/v3/projects => generated 379 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:20:05.540562 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c19473-10d4-4197-94dc-b30a18598585 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:05.552851 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c19473-10d4-4197-94dc-b30a18598585 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:05.615985 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:20:05.624966 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6556ca6-b698-4914-8b5b-8840fa5407c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:05.625716 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6556ca6-b698-4914-8b5b-8840fa5407c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:05.626059 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6556ca6-b698-4914-8b5b-8840fa5407c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:05.626385 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6556ca6-b698-4914-8b5b-8840fa5407c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:05.627185 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6556ca6-b698-4914-8b5b-8840fa5407c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:05.628474 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: WARNING py.warnings [None req-e6556ca6-b698-4914-8b5b-8840fa5407c2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 19:20:05.628474 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: warnings.warn(msg) Jul 25 19:20:05.628474 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: Jul 25 19:20:05.630419 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6556ca6-b698-4914-8b5b-8840fa5407c2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:05.661654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 739/1498] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:20:05 2022] POST /identity/v3/users => generated 290 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:20:05.666221 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67fbdaa-e6c0-4b41-b1e9-5895d12867a3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:05.677875 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67fbdaa-e6c0-4b41-b1e9-5895d12867a3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:05.683456 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79332cbe-5635-494a-9223-c96b519866ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:05.684345 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79332cbe-5635-494a-9223-c96b519866ce admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:05.684726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79332cbe-5635-494a-9223-c96b519866ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:05.685143 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79332cbe-5635-494a-9223-c96b519866ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:05.686033 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79332cbe-5635-494a-9223-c96b519866ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:05.687166 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-79332cbe-5635-494a-9223-c96b519866ce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 19:20:05.687166 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: warnings.warn(msg) Jul 25 19:20:05.687166 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: Jul 25 19:20:05.688390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79332cbe-5635-494a-9223-c96b519866ce admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:05.704844 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 760/1499] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:20:05 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:20:05.710130 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6556ca6-b698-4914-8b5b-8840fa5407c2 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:05.721563 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6556ca6-b698-4914-8b5b-8840fa5407c2 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:05.727527 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec7dc9-c9a6-41b9-b51c-c650a47624b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:05.728292 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec7dc9-c9a6-41b9-b51c-c650a47624b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:05.728616 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec7dc9-c9a6-41b9-b51c-c650a47624b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:05.728935 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec7dc9-c9a6-41b9-b51c-c650a47624b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:05.729817 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec7dc9-c9a6-41b9-b51c-c650a47624b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:05.731228 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: WARNING py.warnings [None req-feec7dc9-c9a6-41b9-b51c-c650a47624b6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 19:20:05.731228 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: warnings.warn(msg) Jul 25 19:20:05.731228 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: Jul 25 19:20:05.732363 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec7dc9-c9a6-41b9-b51c-c650a47624b6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:05.749713 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 740/1500] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:20:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:20:05.754407 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79332cbe-5635-494a-9223-c96b519866ce admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:05.767242 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79332cbe-5635-494a-9223-c96b519866ce admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:05.772948 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacced5b-61c9-4f89-89f8-ca7ae3daf4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:05.773874 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacced5b-61c9-4f89-89f8-ca7ae3daf4b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:05.774163 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacced5b-61c9-4f89-89f8-ca7ae3daf4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:05.774442 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacced5b-61c9-4f89-89f8-ca7ae3daf4b0 admin admin] PATH_INFO: `/v3/projects/175c97f4dcd64a60a145109abb0e9e3a/users/4c2e8beda8644ca780626e02280ba035/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:05.794902 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacced5b-61c9-4f89-89f8-ca7ae3daf4b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=175c97f4dcd64a60a145109abb0e9e3a, user_id=4c2e8beda8644ca780626e02280ba035, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:05.796204 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-bacced5b-61c9-4f89-89f8-ca7ae3daf4b0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 19:20:05.796204 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: warnings.warn(msg) Jul 25 19:20:05.796204 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: Jul 25 19:20:05.798392 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacced5b-61c9-4f89-89f8-ca7ae3daf4b0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:05.815735 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 761/1501] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:20:05 2022] PUT /identity/v3/projects/175c97f4dcd64a60a145109abb0e9e3a/users/4c2e8beda8644ca780626e02280ba035/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:20:05.838343 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecfd41-6a72-4167-98b9-44d74ed6f805 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:05.838756 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecfd41-6a72-4167-98b9-44d74ed6f805 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:05.839086 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecfd41-6a72-4167-98b9-44d74ed6f805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:05.874331 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-84ecfd41-6a72-4167-98b9-44d74ed6f805 None None] MFA Rules not processed for user `4c2e8beda8644ca780626e02280ba035`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:20:05.875847 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-84ecfd41-6a72-4167-98b9-44d74ed6f805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:05.961294 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 741/1502] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:20:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:20:05.977039 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bda1f1-7bd5-4cb9-94d3-5e87074fc040 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:05.977491 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bda1f1-7bd5-4cb9-94d3-5e87074fc040 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:05.977815 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bda1f1-7bd5-4cb9-94d3-5e87074fc040 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:05.979442 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 762/1503] 10.222.0.18 () {54 vars in 964 bytes} [Mon Jul 25 19:20:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:20:05.985120 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc68d1-b6c2-4916-94d0-c3eab1bd15e2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:05.985421 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc68d1-b6c2-4916-94d0-c3eab1bd15e2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:05.985699 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc68d1-b6c2-4916-94d0-c3eab1bd15e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:06.077507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8cdc68d1-b6c2-4916-94d0-c3eab1bd15e2 None None] MFA Rules not processed for user `2aa720a35b9844848abcc42a4c4d6a2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:20:06.078925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8cdc68d1-b6c2-4916-94d0-c3eab1bd15e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:06.177954 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 742/1504] 10.222.0.18 () {58 vars in 1061 bytes} [Mon Jul 25 19:20:05 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:20:06.186168 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bda1f1-7bd5-4cb9-94d3-5e87074fc040 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:06.199159 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bda1f1-7bd5-4cb9-94d3-5e87074fc040 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:06.203155 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9967559a-4290-4a40-a6b1-d9012167d33c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:06.203884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967559a-4290-4a40-a6b1-d9012167d33c service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:06.204170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967559a-4290-4a40-a6b1-d9012167d33c service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:06.204486 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967559a-4290-4a40-a6b1-d9012167d33c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:06.215851 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9967559a-4290-4a40-a6b1-d9012167d33c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:06.217271 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9967559a-4290-4a40-a6b1-d9012167d33c service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:06.229661 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 763/1505] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:20:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:20:06.284693 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2003f-ff4b-4e28-bd2a-9a9d51a42f38 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:06.285132 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2003f-ff4b-4e28-bd2a-9a9d51a42f38 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:06.285524 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2003f-ff4b-4e28-bd2a-9a9d51a42f38 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:06.287059 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 743/1506] 10.222.0.18 () {52 vars in 830 bytes} [Mon Jul 25 19:20:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:20:06.291884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9967559a-4290-4a40-a6b1-d9012167d33c service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:06.370874 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9967559a-4290-4a40-a6b1-d9012167d33c service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:06.376205 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4074497c-2a22-403a-8d0c-413323fa250f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:06.377120 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074497c-2a22-403a-8d0c-413323fa250f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:06.377397 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074497c-2a22-403a-8d0c-413323fa250f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:06.377680 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074497c-2a22-403a-8d0c-413323fa250f admin admin] PATH_INFO: `/v3/users/4c2e8beda8644ca780626e02280ba035` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:06.379080 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4074497c-2a22-403a-8d0c-413323fa250f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c2e8beda8644ca780626e02280ba035)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:06.380244 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-4074497c-2a22-403a-8d0c-413323fa250f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 19:20:06.380244 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: warnings.warn(msg) Jul 25 19:20:06.380244 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: Jul 25 19:20:06.382041 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4074497c-2a22-403a-8d0c-413323fa250f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:06.463128 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4074497c-2a22-403a-8d0c-413323fa250f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c2e8beda8644ca780626e02280ba035'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:06.478752 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4074497c-2a22-403a-8d0c-413323fa250f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c2e8beda8644ca780626e02280ba035'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:06.491549 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4074497c-2a22-403a-8d0c-413323fa250f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c2e8beda8644ca780626e02280ba035'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:06.492487 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4074497c-2a22-403a-8d0c-413323fa250f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c2e8beda8644ca780626e02280ba035'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:06.493229 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4074497c-2a22-403a-8d0c-413323fa250f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c2e8beda8644ca780626e02280ba035'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:06.514024 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:20:06.524984 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 764/1507] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:20:06 2022] DELETE /identity/v3/users/4c2e8beda8644ca780626e02280ba035 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:20:06.529857 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a00829-44c4-4693-ada6-4264b1f9cac4 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:06.530147 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a00829-44c4-4693-ada6-4264b1f9cac4 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:06.530436 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a00829-44c4-4693-ada6-4264b1f9cac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:06.556520 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-70a00829-44c4-4693-ada6-4264b1f9cac4 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:20:06.557792 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-70a00829-44c4-4693-ada6-4264b1f9cac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:06.568595 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:20:06.631686 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 744/1508] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:20:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:20:06.643258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4074497c-2a22-403a-8d0c-413323fa250f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:06.647543 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4074497c-2a22-403a-8d0c-413323fa250f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:06.751176 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4074497c-2a22-403a-8d0c-413323fa250f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:06.756023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da917429-62cb-463e-afd2-5963a5fd0e3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:06.757403 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da917429-62cb-463e-afd2-5963a5fd0e3d service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:06.757814 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da917429-62cb-463e-afd2-5963a5fd0e3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:06.758291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da917429-62cb-463e-afd2-5963a5fd0e3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:06.769033 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da917429-62cb-463e-afd2-5963a5fd0e3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:06.771624 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da917429-62cb-463e-afd2-5963a5fd0e3d service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:06.783968 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 765/1509] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:20:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:20:07.194823 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a00829-44c4-4693-ada6-4264b1f9cac4 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:20:07.196594 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-70a00829-44c4-4693-ada6-4264b1f9cac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:20:07.214429 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a00829-44c4-4693-ada6-4264b1f9cac4 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:20:07.220126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38488a6-56a6-47ad-bde2-ee4b0d7802c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:20:07.221196 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38488a6-56a6-47ad-bde2-ee4b0d7802c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:20:07.221492 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38488a6-56a6-47ad-bde2-ee4b0d7802c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:20:07.221786 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38488a6-56a6-47ad-bde2-ee4b0d7802c8 admin admin] PATH_INFO: `/v3/projects/175c97f4dcd64a60a145109abb0e9e3a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:20:07.223880 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38488a6-56a6-47ad-bde2-ee4b0d7802c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=175c97f4dcd64a60a145109abb0e9e3a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:20:07.225044 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: WARNING py.warnings [None req-a38488a6-56a6-47ad-bde2-ee4b0d7802c8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 19:20:07.225044 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: warnings.warn(msg) Jul 25 19:20:07.225044 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: Jul 25 19:20:07.226275 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38488a6-56a6-47ad-bde2-ee4b0d7802c8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:20:07.319003 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a38488a6-56a6-47ad-bde2-ee4b0d7802c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '175c97f4dcd64a60a145109abb0e9e3a', 'request_id': 'req-a38488a6-56a6-47ad-bde2-ee4b0d7802c8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:07.331713 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a38488a6-56a6-47ad-bde2-ee4b0d7802c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '175c97f4dcd64a60a145109abb0e9e3a', 'request_id': 'req-a38488a6-56a6-47ad-bde2-ee4b0d7802c8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:07.339794 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a38488a6-56a6-47ad-bde2-ee4b0d7802c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '175c97f4dcd64a60a145109abb0e9e3a', 'request_id': 'req-a38488a6-56a6-47ad-bde2-ee4b0d7802c8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:07.362073 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a38488a6-56a6-47ad-bde2-ee4b0d7802c8 admin admin] The token cache is being invalidate because project 175c97f4dcd64a60a145109abb0e9e3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:20:07.362443 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a38488a6-56a6-47ad-bde2-ee4b0d7802c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:07.363224 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a38488a6-56a6-47ad-bde2-ee4b0d7802c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:20:07.365839 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 745/1510] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:20:07 2022] DELETE /identity/v3/projects/175c97f4dcd64a60a145109abb0e9e3a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:20:16.777966 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:20:17.361164 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:22:04.752525 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1352f-e037-497b-af86-2030a8d92ec3 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:22:04.753153 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1352f-e037-497b-af86-2030a8d92ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:22:04.753153 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1352f-e037-497b-af86-2030a8d92ec3 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:22:04.754485 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 766/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 19:22:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:22:04.759789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66877c-b592-4889-a847-aa63b4b35c92 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:22:04.760276 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66877c-b592-4889-a847-aa63b4b35c92 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:22:04.760380 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66877c-b592-4889-a847-aa63b4b35c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:22:04.783819 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ee66877c-b592-4889-a847-aa63b4b35c92 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:22:04.785057 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ee66877c-b592-4889-a847-aa63b4b35c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:22:04.846139 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 746/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 19:22:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:22:05.005117 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d1352f-e037-497b-af86-2030a8d92ec3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:22:05.006804 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-48d1352f-e037-497b-af86-2030a8d92ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:22:05.126837 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d1352f-e037-497b-af86-2030a8d92ec3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:22:05.132570 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4630d997-221b-4e01-aa49-d845d676dce3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:22:05.133236 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630d997-221b-4e01-aa49-d845d676dce3 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:22:05.133526 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630d997-221b-4e01-aa49-d845d676dce3 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:22:05.133805 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630d997-221b-4e01-aa49-d845d676dce3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:22:05.144008 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4630d997-221b-4e01-aa49-d845d676dce3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:22:05.145735 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4630d997-221b-4e01-aa49-d845d676dce3 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:22:05.156269 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 767/1513] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:22:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:22:06.032507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68e9a8-f675-4eab-bedd-48e4de3c0ae1 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:22:06.033093 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68e9a8-f675-4eab-bedd-48e4de3c0ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:22:06.033307 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68e9a8-f675-4eab-bedd-48e4de3c0ae1 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:22:06.034752 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 747/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 19:22:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:22:06.040170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3208da-0296-402c-b807-cc9e5c8e9f7c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:22:06.040717 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3208da-0296-402c-b807-cc9e5c8e9f7c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:22:06.040717 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3208da-0296-402c-b807-cc9e5c8e9f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:22:06.062162 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0b3208da-0296-402c-b807-cc9e5c8e9f7c None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:22:06.063728 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0b3208da-0296-402c-b807-cc9e5c8e9f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:22:06.072251 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 768/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 19:22:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:22:06.279984 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be68e9a8-f675-4eab-bedd-48e4de3c0ae1 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:22:06.297136 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be68e9a8-f675-4eab-bedd-48e4de3c0ae1 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:22:06.303271 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270651b-9bf5-4983-93ea-d7c449861efd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:22:06.303967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270651b-9bf5-4983-93ea-d7c449861efd service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:22:06.304249 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270651b-9bf5-4983-93ea-d7c449861efd service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:22:06.304523 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270651b-9bf5-4983-93ea-d7c449861efd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:22:06.313963 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a270651b-9bf5-4983-93ea-d7c449861efd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:22:06.315609 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a270651b-9bf5-4983-93ea-d7c449861efd service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:22:06.325395 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 748/1516] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:22:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:22:07.068915 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6921ab-8e7e-4ff5-a428-c8a1fbe7ed46 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:22:07.069784 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6921ab-8e7e-4ff5-a428-c8a1fbe7ed46 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:22:07.069784 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6921ab-8e7e-4ff5-a428-c8a1fbe7ed46 None None] PATH_INFO: `/v3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:22:07.071036 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 769/1517] 10.222.0.119 () {54 vars in 926 bytes} [Mon Jul 25 19:22:07 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:22:07.080393 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e82fe7-2eb9-4e2d-8248-8e612bd23bd1 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:22:07.081161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e82fe7-2eb9-4e2d-8248-8e612bd23bd1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:22:07.081161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e82fe7-2eb9-4e2d-8248-8e612bd23bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:22:07.144664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f4e82fe7-2eb9-4e2d-8248-8e612bd23bd1 None None] MFA Rules not processed for user `0422d6695bf14925bf0f993cb16e64ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:22:07.146030 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f4e82fe7-2eb9-4e2d-8248-8e612bd23bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:22:07.257106 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 749/1518] 10.222.0.119 () {58 vars in 1016 bytes} [Mon Jul 25 19:22:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:22:16.056468 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=49 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:22:17.251477 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=77 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:22:17.677105 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf00c0d-768e-4028-8bfd-3be05587dfc7 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:22:17.677979 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf00c0d-768e-4028-8bfd-3be05587dfc7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:22:17.677979 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf00c0d-768e-4028-8bfd-3be05587dfc7 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:22:17.679121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 770/1519] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 19:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:22:17.684940 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d675a2-1705-4e03-a56c-b1546c8760e7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:22:17.685272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d675a2-1705-4e03-a56c-b1546c8760e7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:22:17.685538 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d675a2-1705-4e03-a56c-b1546c8760e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:22:17.709298 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-23d675a2-1705-4e03-a56c-b1546c8760e7 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:22:17.710565 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-23d675a2-1705-4e03-a56c-b1546c8760e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:22:17.719336 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 750/1520] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 19:22:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:22:17.880051 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf00c0d-768e-4028-8bfd-3be05587dfc7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:22:17.895426 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf00c0d-768e-4028-8bfd-3be05587dfc7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:22:17.901480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9700fcb5-b1d6-4220-8721-3b6ba3fa2b25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:22:17.902147 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700fcb5-b1d6-4220-8721-3b6ba3fa2b25 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:22:17.902436 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700fcb5-b1d6-4220-8721-3b6ba3fa2b25 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:22:17.902718 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700fcb5-b1d6-4220-8721-3b6ba3fa2b25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:22:17.912397 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9700fcb5-b1d6-4220-8721-3b6ba3fa2b25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:22:17.914046 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9700fcb5-b1d6-4220-8721-3b6ba3fa2b25 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:22:17.925668 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 771/1521] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:22:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:22:18.843637 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b158a3d-91a7-4188-9009-4a4b5455752d None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:22:18.844853 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b158a3d-91a7-4188-9009-4a4b5455752d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:22:18.844853 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b158a3d-91a7-4188-9009-4a4b5455752d None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:22:18.845806 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 751/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 19:22:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:22:18.853798 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e4c28-60a6-4244-95d6-1741bf1ce842 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:22:18.854375 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e4c28-60a6-4244-95d6-1741bf1ce842 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:22:18.854817 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e4c28-60a6-4244-95d6-1741bf1ce842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:22:18.885451 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-bf3e4c28-60a6-4244-95d6-1741bf1ce842 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:22:18.886926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bf3e4c28-60a6-4244-95d6-1741bf1ce842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:22:18.895757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 772/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 19:22:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:22:19.038401 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5b14d-8957-4f53-83cc-c771b4ec5d11 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:22:19.038630 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5b14d-8957-4f53-83cc-c771b4ec5d11 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:22:19.038877 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db5b14d-8957-4f53-83cc-c771b4ec5d11 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:22:19.040301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 752/1524] 10.222.0.18 () {54 vars in 961 bytes} [Mon Jul 25 19:22:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:22:19.045654 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a1016c-a371-43bf-b6d7-6283bee1698d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:22:19.045950 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a1016c-a371-43bf-b6d7-6283bee1698d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:22:19.046232 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a1016c-a371-43bf-b6d7-6283bee1698d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:22:19.050806 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db5b14d-8957-4f53-83cc-c771b4ec5d11 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:22:19.067145 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-83a1016c-a371-43bf-b6d7-6283bee1698d None None] MFA Rules not processed for user `0422d6695bf14925bf0f993cb16e64ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:22:19.067405 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db5b14d-8957-4f53-83cc-c771b4ec5d11 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:22:19.068414 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-83a1016c-a371-43bf-b6d7-6283bee1698d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:22:19.073408 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd83926-4a61-4c80-a5c8-b98fd767d63e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:22:19.074080 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd83926-4a61-4c80-a5c8-b98fd767d63e service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:22:19.074411 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd83926-4a61-4c80-a5c8-b98fd767d63e service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:22:19.074688 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd83926-4a61-4c80-a5c8-b98fd767d63e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:22:19.079071 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 773/1525] 10.222.0.18 () {58 vars in 1058 bytes} [Mon Jul 25 19:22:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:22:19.085680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd83926-4a61-4c80-a5c8-b98fd767d63e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:22:19.086674 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a1016c-a371-43bf-b6d7-6283bee1698d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:22:19.087300 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd83926-4a61-4c80-a5c8-b98fd767d63e service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:22:19.098273 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a1016c-a371-43bf-b6d7-6283bee1698d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:22:19.098657 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 753/1526] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:22:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:22:19.104657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326835cb-f512-45fc-b937-f301fe74dc22 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0422d6695bf14925bf0f993cb16e64ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:22:19.105012 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326835cb-f512-45fc-b937-f301fe74dc22 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:22:19.105291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326835cb-f512-45fc-b937-f301fe74dc22 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:22:19.105578 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326835cb-f512-45fc-b937-f301fe74dc22 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:22:19.115910 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326835cb-f512-45fc-b937-f301fe74dc22 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:22:19.117570 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326835cb-f512-45fc-b937-f301fe74dc22 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:22:19.130289 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 774/1527] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:22:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:22:23.381573 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165e1bd-5ca2-4075-95c3-57de7aec7050 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:22:23.382136 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165e1bd-5ca2-4075-95c3-57de7aec7050 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:22:23.382595 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165e1bd-5ca2-4075-95c3-57de7aec7050 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:22:23.385362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 754/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 19:22:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:22:23.394592 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0257fa68-2c3d-4205-a9f6-101cd9c7b8d3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:22:23.395378 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0257fa68-2c3d-4205-a9f6-101cd9c7b8d3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:22:23.395378 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0257fa68-2c3d-4205-a9f6-101cd9c7b8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:22:23.418822 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0257fa68-2c3d-4205-a9f6-101cd9c7b8d3 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:22:23.420272 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0257fa68-2c3d-4205-a9f6-101cd9c7b8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:22:23.430086 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 775/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 19:22:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:22:23.515030 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3165e1bd-5ca2-4075-95c3-57de7aec7050 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:22:23.517823 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3165e1bd-5ca2-4075-95c3-57de7aec7050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:22:23.589069 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3165e1bd-5ca2-4075-95c3-57de7aec7050 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:22:23.595257 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6623ef7-64d4-4865-8807-1a0b91b6e4e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:22:23.596437 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6623ef7-64d4-4865-8807-1a0b91b6e4e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:22:23.596950 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6623ef7-64d4-4865-8807-1a0b91b6e4e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:22:23.597459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6623ef7-64d4-4865-8807-1a0b91b6e4e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:22:23.611023 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6623ef7-64d4-4865-8807-1a0b91b6e4e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:22:23.613062 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6623ef7-64d4-4865-8807-1a0b91b6e4e4 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:22:23.625513 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 755/1530] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:22:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:22:26.335864 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d708098-c9fe-4eaa-b8fc-7281e26098c1 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:22:26.336635 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d708098-c9fe-4eaa-b8fc-7281e26098c1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:22:26.336635 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d708098-c9fe-4eaa-b8fc-7281e26098c1 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:22:26.338557 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 776/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 19:22:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:22:26.349251 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5022863-a777-4209-9a62-a0f063389422 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:22:26.350210 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5022863-a777-4209-9a62-a0f063389422 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:22:26.350402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5022863-a777-4209-9a62-a0f063389422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:22:26.394759 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c5022863-a777-4209-9a62-a0f063389422 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:22:26.396887 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c5022863-a777-4209-9a62-a0f063389422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:22:26.417036 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 756/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 19:22:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:22:26.483833 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d708098-c9fe-4eaa-b8fc-7281e26098c1 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:22:26.505154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d708098-c9fe-4eaa-b8fc-7281e26098c1 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:22:26.512310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b1820-c938-40c0-8c55-5e89e7be681b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:22:26.512679 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b1820-c938-40c0-8c55-5e89e7be681b service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:22:26.513136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b1820-c938-40c0-8c55-5e89e7be681b service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:22:26.513537 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b1820-c938-40c0-8c55-5e89e7be681b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:22:26.534012 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937b1820-c938-40c0-8c55-5e89e7be681b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:22:26.537816 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937b1820-c938-40c0-8c55-5e89e7be681b service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:22:26.558452 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 777/1533] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:22:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:22:31.231712 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7e1ea-75d1-467f-a918-ae2802614c18 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:22:31.232549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7e1ea-75d1-467f-a918-ae2802614c18 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:22:31.232784 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7e1ea-75d1-467f-a918-ae2802614c18 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:22:31.235665 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 757/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 19:22:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:22:31.244481 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e6ab0-7232-4472-ada0-13c2d2d4f6df None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:22:31.245367 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e6ab0-7232-4472-ada0-13c2d2d4f6df None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:22:31.245515 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e6ab0-7232-4472-ada0-13c2d2d4f6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:22:31.287016 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-476e6ab0-7232-4472-ada0-13c2d2d4f6df None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:22:31.289896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-476e6ab0-7232-4472-ada0-13c2d2d4f6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:22:31.305554 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 778/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 19:22:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:22:31.898040 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7e1ea-75d1-467f-a918-ae2802614c18 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:22:31.904682 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3ce7e1ea-75d1-467f-a918-ae2802614c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:22:32.000702 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7e1ea-75d1-467f-a918-ae2802614c18 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:22:32.005129 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23576b86-afe1-46d6-9e5b-6b1cd23ea2cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:22:32.005816 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23576b86-afe1-46d6-9e5b-6b1cd23ea2cc service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:22:32.006115 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23576b86-afe1-46d6-9e5b-6b1cd23ea2cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:22:32.006395 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23576b86-afe1-46d6-9e5b-6b1cd23ea2cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:22:32.024888 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23576b86-afe1-46d6-9e5b-6b1cd23ea2cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:22:32.027756 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23576b86-afe1-46d6-9e5b-6b1cd23ea2cc service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:22:32.047849 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 758/1536] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:22:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:22:34.821901 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bc3fc-37f0-4593-bcd4-49c0ea29f9bb None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:22:34.825729 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bc3fc-37f0-4593-bcd4-49c0ea29f9bb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:22:34.826023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bc3fc-37f0-4593-bcd4-49c0ea29f9bb None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:22:34.828326 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 779/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 19:22:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:22:34.837807 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e82bb2-6add-48bf-a188-66a5b5594216 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:22:34.838731 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e82bb2-6add-48bf-a188-66a5b5594216 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:22:34.838731 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e82bb2-6add-48bf-a188-66a5b5594216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:22:34.883376 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d2e82bb2-6add-48bf-a188-66a5b5594216 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:22:34.885684 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d2e82bb2-6add-48bf-a188-66a5b5594216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:22:34.901543 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 759/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 19:22:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:22:35.465988 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4bc3fc-37f0-4593-bcd4-49c0ea29f9bb None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:22:35.470288 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bf4bc3fc-37f0-4593-bcd4-49c0ea29f9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:22:35.494451 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4bc3fc-37f0-4593-bcd4-49c0ea29f9bb None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:22:35.501674 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9a5a12-bbdc-49a5-8d2d-538e458e5255 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:22:35.502759 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a5a12-bbdc-49a5-8d2d-538e458e5255 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:22:35.503260 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a5a12-bbdc-49a5-8d2d-538e458e5255 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:22:35.503832 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a5a12-bbdc-49a5-8d2d-538e458e5255 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:22:35.522506 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9a5a12-bbdc-49a5-8d2d-538e458e5255 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:22:35.525071 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9a5a12-bbdc-49a5-8d2d-538e458e5255 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:22:35.543948 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 780/1539] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:22:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:22:44.873578 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=67 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:22:45.535927 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=36 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:24.592037 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a755475-fd83-416b-a45e-a54d8e935f4f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:24.594944 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a755475-fd83-416b-a45e-a54d8e935f4f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:24.594944 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a755475-fd83-416b-a45e-a54d8e935f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:24.621224 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0a755475-fd83-416b-a45e-a54d8e935f4f None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:24.622548 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0a755475-fd83-416b-a45e-a54d8e935f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:24.639856 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 760/1540] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:24.655351 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be228458-815c-44a1-821a-930b93064831 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:24.656362 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be228458-815c-44a1-821a-930b93064831 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:24.656362 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be228458-815c-44a1-821a-930b93064831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:24.683011 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048eaeed-084c-47ed-84de-c521d68b53d0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:24.683579 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048eaeed-084c-47ed-84de-c521d68b53d0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:24.684092 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048eaeed-084c-47ed-84de-c521d68b53d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:24.716310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-be228458-815c-44a1-821a-930b93064831 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:24.723718 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-be228458-815c-44a1-821a-930b93064831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:24.735993 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-048eaeed-084c-47ed-84de-c521d68b53d0 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:24.738264 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-048eaeed-084c-47ed-84de-c521d68b53d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:24.751678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 781/1541] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:24:24.755338 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be228458-815c-44a1-821a-930b93064831 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:24.766622 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 761/1542] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:24.774895 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be228458-815c-44a1-821a-930b93064831 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:24.777573 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08cd91-9cc5-404a-b877-983e80ef31b3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:24.778351 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08cd91-9cc5-404a-b877-983e80ef31b3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:24.778932 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08cd91-9cc5-404a-b877-983e80ef31b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:24.797037 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c785ff7-4ed0-4df4-8d62-cc47cbf53368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:24.798192 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c785ff7-4ed0-4df4-8d62-cc47cbf53368 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:24.798680 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c785ff7-4ed0-4df4-8d62-cc47cbf53368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:24.799183 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c785ff7-4ed0-4df4-8d62-cc47cbf53368 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:24.801276 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c785ff7-4ed0-4df4-8d62-cc47cbf53368 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:24:24.801817 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c785ff7-4ed0-4df4-8d62-cc47cbf53368 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:24.803817 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-2c785ff7-4ed0-4df4-8d62-cc47cbf53368 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 19:24:24.803817 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: warnings.warn(msg) Jul 25 19:24:24.803817 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: Jul 25 19:24:24.805637 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c785ff7-4ed0-4df4-8d62-cc47cbf53368 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:24.843915 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 782/1543] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:24:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:24:24.845388 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ce08cd91-9cc5-404a-b877-983e80ef31b3 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:24.847427 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ce08cd91-9cc5-404a-b877-983e80ef31b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:24.857545 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c785ff7-4ed0-4df4-8d62-cc47cbf53368 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:24.875775 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 762/1544] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:24.878137 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf854208-a2a5-4416-ad7e-852d715f7829 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:24.878769 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf854208-a2a5-4416-ad7e-852d715f7829 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:24.879214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf854208-a2a5-4416-ad7e-852d715f7829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:24.898253 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c785ff7-4ed0-4df4-8d62-cc47cbf53368 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:24.928825 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeda61a-6d22-441d-b644-97a555d02a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:24.929925 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeda61a-6d22-441d-b644-97a555d02a24 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:24.930391 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeda61a-6d22-441d-b644-97a555d02a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:24.931105 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeda61a-6d22-441d-b644-97a555d02a24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:24.932465 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeda61a-6d22-441d-b644-97a555d02a24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:24.937250 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeda61a-6d22-441d-b644-97a555d02a24 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:24.945611 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-cf854208-a2a5-4416-ad7e-852d715f7829 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:24.947731 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cf854208-a2a5-4416-ad7e-852d715f7829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:24.979292 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 763/1545] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:24.980739 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf854208-a2a5-4416-ad7e-852d715f7829 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:25.011363 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf854208-a2a5-4416-ad7e-852d715f7829 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:25.016316 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 783/1546] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:24:24 2022] POST /identity/v3/projects => generated 389 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:24:25.022016 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c4acb-6cc7-4c88-a4f1-f10f70330e7a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:25.022502 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c4acb-6cc7-4c88-a4f1-f10f70330e7a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:25.022969 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c4acb-6cc7-4c88-a4f1-f10f70330e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:25.027012 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d7faf-56ae-4eec-a206-130fd8542192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:25.028130 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d7faf-56ae-4eec-a206-130fd8542192 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:25.028595 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d7faf-56ae-4eec-a206-130fd8542192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:25.029035 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d7faf-56ae-4eec-a206-130fd8542192 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:25.030300 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588d7faf-56ae-4eec-a206-130fd8542192 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:24:25.030800 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588d7faf-56ae-4eec-a206-130fd8542192 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:25.033844 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588d7faf-56ae-4eec-a206-130fd8542192 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:25.072453 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 764/1547] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:24:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:24:25.075503 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b72c4acb-6cc7-4c88-a4f1-f10f70330e7a None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:25.076792 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2600e11-7be7-46a1-95ea-a30fd39762c3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:25.077156 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2600e11-7be7-46a1-95ea-a30fd39762c3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:25.078310 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2600e11-7be7-46a1-95ea-a30fd39762c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:25.079462 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b72c4acb-6cc7-4c88-a4f1-f10f70330e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:25.130052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c2600e11-7be7-46a1-95ea-a30fd39762c3 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:25.133255 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c2600e11-7be7-46a1-95ea-a30fd39762c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:25.219427 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 784/1548] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:25.221179 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c4acb-6cc7-4c88-a4f1-f10f70330e7a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:25.239604 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c4acb-6cc7-4c88-a4f1-f10f70330e7a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:25.244002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 765/1549] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:25.251701 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3eccd2-d421-463c-a784-2f5288116687 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:25.251701 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3eccd2-d421-463c-a784-2f5288116687 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:25.251701 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3eccd2-d421-463c-a784-2f5288116687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:25.255547 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec176a-33e0-4bea-a16b-3a2350b59b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:25.256793 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec176a-33e0-4bea-a16b-3a2350b59b68 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:25.257313 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec176a-33e0-4bea-a16b-3a2350b59b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:25.257831 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec176a-33e0-4bea-a16b-3a2350b59b68 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:25.259185 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec176a-33e0-4bea-a16b-3a2350b59b68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:25.261191 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-94ec176a-33e0-4bea-a16b-3a2350b59b68 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 19:24:25.261191 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: warnings.warn(msg) Jul 25 19:24:25.261191 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: Jul 25 19:24:25.264566 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec176a-33e0-4bea-a16b-3a2350b59b68 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:25.287385 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-de3eccd2-d421-463c-a784-2f5288116687 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:25.289441 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-de3eccd2-d421-463c-a784-2f5288116687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:25.303413 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 766/1550] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:25.305291 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3eccd2-d421-463c-a784-2f5288116687 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:25.315818 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 785/1551] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:24:25 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:24:25.326977 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4f27d-7a76-492b-bd29-75ee724a4c64 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:25.327298 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3eccd2-d421-463c-a784-2f5288116687 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:25.328138 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4f27d-7a76-492b-bd29-75ee724a4c64 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:25.329160 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4f27d-7a76-492b-bd29-75ee724a4c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:25.336630 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920460a6-c3ed-4858-a5c9-a07e7a15e162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:25.337861 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920460a6-c3ed-4858-a5c9-a07e7a15e162 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:25.338326 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920460a6-c3ed-4858-a5c9-a07e7a15e162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:25.338768 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920460a6-c3ed-4858-a5c9-a07e7a15e162 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:25.341217 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920460a6-c3ed-4858-a5c9-a07e7a15e162 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:25.343272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: WARNING py.warnings [None req-920460a6-c3ed-4858-a5c9-a07e7a15e162 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 19:24:25.343272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: warnings.warn(msg) Jul 25 19:24:25.343272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: Jul 25 19:24:25.346403 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920460a6-c3ed-4858-a5c9-a07e7a15e162 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:25.374835 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-98a4f27d-7a76-492b-bd29-75ee724a4c64 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:25.377333 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-98a4f27d-7a76-492b-bd29-75ee724a4c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:25.396064 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 786/1552] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:25.399548 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b0679-f65f-4de7-8016-9d2fd5830835 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:25.399880 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 767/1553] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:24:25 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:24:25.400199 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b0679-f65f-4de7-8016-9d2fd5830835 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:25.400907 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b0679-f65f-4de7-8016-9d2fd5830835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:25.402998 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920460a6-c3ed-4858-a5c9-a07e7a15e162 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:25.446294 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-414b0679-f65f-4de7-8016-9d2fd5830835 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:25.448534 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-414b0679-f65f-4de7-8016-9d2fd5830835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:25.517453 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920460a6-c3ed-4858-a5c9-a07e7a15e162 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:25.525111 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b9bed-3c2e-43dd-9cd1-b92335862265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:25.525976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b9bed-3c2e-43dd-9cd1-b92335862265 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:25.526267 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b9bed-3c2e-43dd-9cd1-b92335862265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:25.528176 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b9bed-3c2e-43dd-9cd1-b92335862265 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:25.528176 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5b9bed-3c2e-43dd-9cd1-b92335862265 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:24:25.528497 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5b9bed-3c2e-43dd-9cd1-b92335862265 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:25.530378 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5b9bed-3c2e-43dd-9cd1-b92335862265 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:25.553561 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 768/1554] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:24:25.556039 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c0d27-659d-44ea-a500-0a70325f40ab None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:25.556563 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c0d27-659d-44ea-a500-0a70325f40ab None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:25.557049 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c0d27-659d-44ea-a500-0a70325f40ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:25.579845 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 787/1555] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:25.581699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414b0679-f65f-4de7-8016-9d2fd5830835 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:25.586049 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3a4c0d27-659d-44ea-a500-0a70325f40ab None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:25.587373 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3a4c0d27-659d-44ea-a500-0a70325f40ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:25.596962 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 769/1556] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:25.598597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194a8ad-64c3-4161-a7a1-93f0e48d55ba None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:25.598977 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194a8ad-64c3-4161-a7a1-93f0e48d55ba None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:25.599274 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194a8ad-64c3-4161-a7a1-93f0e48d55ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:25.600482 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414b0679-f65f-4de7-8016-9d2fd5830835 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:25.609481 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbffdbb9-97c6-4643-a20b-1b28bf2c8d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:25.610657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbffdbb9-97c6-4643-a20b-1b28bf2c8d0b admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:25.611175 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbffdbb9-97c6-4643-a20b-1b28bf2c8d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:25.611709 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbffdbb9-97c6-4643-a20b-1b28bf2c8d0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:25.613103 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbffdbb9-97c6-4643-a20b-1b28bf2c8d0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:25.616292 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbffdbb9-97c6-4643-a20b-1b28bf2c8d0b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:25.625475 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4194a8ad-64c3-4161-a7a1-93f0e48d55ba None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:25.628802 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4194a8ad-64c3-4161-a7a1-93f0e48d55ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:25.639473 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 788/1557] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:24:25 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:24:25.639473 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbffdbb9-97c6-4643-a20b-1b28bf2c8d0b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:25.640294 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 770/1558] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:25.641812 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7e38c-423e-4142-aac4-ec5efadb4639 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:25.642125 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7e38c-423e-4142-aac4-ec5efadb4639 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:25.642385 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7e38c-423e-4142-aac4-ec5efadb4639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:25.653644 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbffdbb9-97c6-4643-a20b-1b28bf2c8d0b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:25.660186 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e2671-d9c6-4e1d-8c1d-c1f69a15d19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:25.661368 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e2671-d9c6-4e1d-8c1d-c1f69a15d19c admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:25.661763 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e2671-d9c6-4e1d-8c1d-c1f69a15d19c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:25.662102 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e2671-d9c6-4e1d-8c1d-c1f69a15d19c admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:25.665107 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075e2671-d9c6-4e1d-8c1d-c1f69a15d19c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:25.669221 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075e2671-d9c6-4e1d-8c1d-c1f69a15d19c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:25.674655 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-90b7e38c-423e-4142-aac4-ec5efadb4639 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:25.676047 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-90b7e38c-423e-4142-aac4-ec5efadb4639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:25.694127 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 771/1559] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:25.697058 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be2a9d8-f51f-4f63-96a9-13135d072eb2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:25.697058 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be2a9d8-f51f-4f63-96a9-13135d072eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:25.698122 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be2a9d8-f51f-4f63-96a9-13135d072eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:25.708169 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 789/1560] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:24:25 2022] POST /identity/v3/users => generated 293 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:24:25.710343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e2671-d9c6-4e1d-8c1d-c1f69a15d19c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:25.716385 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:25.724479 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3be2a9d8-f51f-4f63-96a9-13135d072eb2 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:25.726599 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e2671-d9c6-4e1d-8c1d-c1f69a15d19c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:25.727300 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3be2a9d8-f51f-4f63-96a9-13135d072eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:25.733191 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f8a70-e9e0-46f7-863e-2e2c3587eeb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:25.734128 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f8a70-e9e0-46f7-863e-2e2c3587eeb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:25.734683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f8a70-e9e0-46f7-863e-2e2c3587eeb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:25.735118 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f8a70-e9e0-46f7-863e-2e2c3587eeb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:25.736168 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f8a70-e9e0-46f7-863e-2e2c3587eeb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:25.738321 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f8a70-e9e0-46f7-863e-2e2c3587eeb2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:25.739113 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 772/1561] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:25.740283 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be2a9d8-f51f-4f63-96a9-13135d072eb2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:25.749072 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:25.757109 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be2a9d8-f51f-4f63-96a9-13135d072eb2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:25.766914 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d2cf6-30bd-4dd7-bea8-608fcf66efcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:25.768495 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d2cf6-30bd-4dd7-bea8-608fcf66efcc admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:25.768495 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d2cf6-30bd-4dd7-bea8-608fcf66efcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:25.768978 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d2cf6-30bd-4dd7-bea8-608fcf66efcc admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:25.770735 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0d2cf6-30bd-4dd7-bea8-608fcf66efcc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:24:25.771315 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0d2cf6-30bd-4dd7-bea8-608fcf66efcc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:25.774477 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0d2cf6-30bd-4dd7-bea8-608fcf66efcc admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:25.783900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 790/1562] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:24:25 2022] POST /identity/v3/projects => generated 365 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:24:25.785664 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f8a70-e9e0-46f7-863e-2e2c3587eeb2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:25.798914 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 773/1563] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:24:25.800044 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d2cf6-30bd-4dd7-bea8-608fcf66efcc admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:25.819085 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d2cf6-30bd-4dd7-bea8-608fcf66efcc admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:25.884185 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f8a70-e9e0-46f7-863e-2e2c3587eeb2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:25.892865 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f34d559-b683-452d-9e07-866939db0f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:25.893984 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34d559-b683-452d-9e07-866939db0f45 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:25.894473 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34d559-b683-452d-9e07-866939db0f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:25.895026 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34d559-b683-452d-9e07-866939db0f45 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:25.896215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f34d559-b683-452d-9e07-866939db0f45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:24:25.896730 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f34d559-b683-452d-9e07-866939db0f45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:25.899153 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f34d559-b683-452d-9e07-866939db0f45 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:25.910483 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd1f0a-905d-4da5-8c9e-5fb21237af2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:25.911904 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd1f0a-905d-4da5-8c9e-5fb21237af2a admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:25.912348 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd1f0a-905d-4da5-8c9e-5fb21237af2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:25.912844 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd1f0a-905d-4da5-8c9e-5fb21237af2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:25.913953 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd1f0a-905d-4da5-8c9e-5fb21237af2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:25.916260 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 791/1564] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:24:25.918128 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd1f0a-905d-4da5-8c9e-5fb21237af2a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:25.924023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e57e30-e60c-4084-9013-34279f26a4de None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:25.924490 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e57e30-e60c-4084-9013-34279f26a4de None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:25.924989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e57e30-e60c-4084-9013-34279f26a4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:25.941271 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 774/1565] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:24:25 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:24:25.942712 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd1f0a-905d-4da5-8c9e-5fb21237af2a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:25.954528 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-54e57e30-e60c-4084-9013-34279f26a4de None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:25.955929 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-54e57e30-e60c-4084-9013-34279f26a4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:25.958378 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd1f0a-905d-4da5-8c9e-5fb21237af2a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:25.968208 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862eca58-4cc8-4cf3-8c61-443031c4a814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:25.968959 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862eca58-4cc8-4cf3-8c61-443031c4a814 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:25.969220 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862eca58-4cc8-4cf3-8c61-443031c4a814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:25.969498 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862eca58-4cc8-4cf3-8c61-443031c4a814 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:25.970437 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862eca58-4cc8-4cf3-8c61-443031c4a814 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:24:25.970851 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862eca58-4cc8-4cf3-8c61-443031c4a814 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:25.971046 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 792/1566] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:25.977246 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e57e30-e60c-4084-9013-34279f26a4de None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:25.977406 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862eca58-4cc8-4cf3-8c61-443031c4a814 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:25.988012 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e57e30-e60c-4084-9013-34279f26a4de None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:25.992200 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 775/1567] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:24:25.993651 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74230796-2905-448e-a286-81f3d11e4902 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:25.993870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74230796-2905-448e-a286-81f3d11e4902 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:25.994161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74230796-2905-448e-a286-81f3d11e4902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:26.005832 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d2fcda-f167-4029-a678-4179737a8b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:26.006920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d2fcda-f167-4029-a678-4179737a8b2d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:26.007417 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d2fcda-f167-4029-a678-4179737a8b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:26.007981 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d2fcda-f167-4029-a678-4179737a8b2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:26.009404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d2fcda-f167-4029-a678-4179737a8b2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:26.013159 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d2fcda-f167-4029-a678-4179737a8b2d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:26.018682 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-74230796-2905-448e-a286-81f3d11e4902 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:26.019990 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-74230796-2905-448e-a286-81f3d11e4902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:26.031499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 793/1568] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:24:25 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:24:26.032514 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 776/1569] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:26.033797 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74230796-2905-448e-a286-81f3d11e4902 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:26.038387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d2fcda-f167-4029-a678-4179737a8b2d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:26.051989 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74230796-2905-448e-a286-81f3d11e4902 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:26.056140 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d2fcda-f167-4029-a678-4179737a8b2d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:26.062162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad4925-2055-4c93-b4e7-8c91ad49f28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:26.063292 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad4925-2055-4c93-b4e7-8c91ad49f28e admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:26.063834 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad4925-2055-4c93-b4e7-8c91ad49f28e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:26.064094 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90128551-df51-4ca8-b3fa-171d7a520494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:26.064319 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad4925-2055-4c93-b4e7-8c91ad49f28e admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:26.065128 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90128551-df51-4ca8-b3fa-171d7a520494 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:26.065626 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ad4925-2055-4c93-b4e7-8c91ad49f28e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:26.065806 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90128551-df51-4ca8-b3fa-171d7a520494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:26.066225 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90128551-df51-4ca8-b3fa-171d7a520494 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:26.067487 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90128551-df51-4ca8-b3fa-171d7a520494 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:26.069428 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ad4925-2055-4c93-b4e7-8c91ad49f28e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:26.071030 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90128551-df51-4ca8-b3fa-171d7a520494 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:26.110348 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 794/1570] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:24:26 2022] POST /identity/v3/users => generated 283 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:24:26.112900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90128551-df51-4ca8-b3fa-171d7a520494 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:26.119106 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 777/1571] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:24:26 2022] POST /identity/v3/projects => generated 373 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:24:26.119106 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad4925-2055-4c93-b4e7-8c91ad49f28e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:26.132675 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90128551-df51-4ca8-b3fa-171d7a520494 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:26.218993 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad4925-2055-4c93-b4e7-8c91ad49f28e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:26.224872 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044745b1-58a6-4c04-b5aa-0a843595131c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:26.226081 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044745b1-58a6-4c04-b5aa-0a843595131c admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:26.226568 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044745b1-58a6-4c04-b5aa-0a843595131c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:26.227023 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044745b1-58a6-4c04-b5aa-0a843595131c admin admin] PATH_INFO: `/v3/projects/5ef063ca1eb846d78ff8901d2a3fc0a3/users/ac2e584f88cd46f280b7fe04aa1382fd/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:26.232226 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136405c-d33a-497e-b5c5-ce019908175c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:26.233118 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136405c-d33a-497e-b5c5-ce019908175c admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:26.233509 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136405c-d33a-497e-b5c5-ce019908175c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:26.233878 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136405c-d33a-497e-b5c5-ce019908175c admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:26.234731 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c136405c-d33a-497e-b5c5-ce019908175c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:26.237208 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c136405c-d33a-497e-b5c5-ce019908175c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:26.248194 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044745b1-58a6-4c04-b5aa-0a843595131c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ef063ca1eb846d78ff8901d2a3fc0a3, user_id=ac2e584f88cd46f280b7fe04aa1382fd, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:26.249365 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: WARNING py.warnings [None req-044745b1-58a6-4c04-b5aa-0a843595131c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 19:24:26.249365 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: warnings.warn(msg) Jul 25 19:24:26.249365 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: Jul 25 19:24:26.253386 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044745b1-58a6-4c04-b5aa-0a843595131c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:26.274516 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 778/1572] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:24:26 2022] PUT /identity/v3/projects/5ef063ca1eb846d78ff8901d2a3fc0a3/users/ac2e584f88cd46f280b7fe04aa1382fd/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:24:26.275398 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044745b1-58a6-4c04-b5aa-0a843595131c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:26.287235 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 795/1573] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:24:26 2022] POST /identity/v3/projects => generated 393 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:24:26.291099 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136405c-d33a-497e-b5c5-ce019908175c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:26.318595 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:26.344923 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:26.386357 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044745b1-58a6-4c04-b5aa-0a843595131c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:26.387163 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136405c-d33a-497e-b5c5-ce019908175c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:26.391994 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a786f-b3b5-44db-8bae-2dfbd853c21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:26.392712 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a786f-b3b5-44db-8bae-2dfbd853c21d admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:26.393003 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a786f-b3b5-44db-8bae-2dfbd853c21d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:26.393277 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a786f-b3b5-44db-8bae-2dfbd853c21d admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:26.394223 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370a786f-b3b5-44db-8bae-2dfbd853c21d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:26.395397 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98b87e-a515-458a-b52c-a5cb6c123a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:26.396409 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370a786f-b3b5-44db-8bae-2dfbd853c21d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:26.396602 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98b87e-a515-458a-b52c-a5cb6c123a85 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:26.397100 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98b87e-a515-458a-b52c-a5cb6c123a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:26.397594 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98b87e-a515-458a-b52c-a5cb6c123a85 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:26.398971 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc98b87e-a515-458a-b52c-a5cb6c123a85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:24:26.399546 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc98b87e-a515-458a-b52c-a5cb6c123a85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:26.403454 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc98b87e-a515-458a-b52c-a5cb6c123a85 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:26.434613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 779/1574] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:24:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:24:26.434613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98b87e-a515-458a-b52c-a5cb6c123a85 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:26.434896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 796/1575] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:24:26 2022] POST /identity/v3/projects => generated 405 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:24:26.434896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a786f-b3b5-44db-8bae-2dfbd853c21d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:26.447173 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98b87e-a515-458a-b52c-a5cb6c123a85 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:26.543329 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13788c0-8967-41ed-b2b5-b048e1f789f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:26.544539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13788c0-8967-41ed-b2b5-b048e1f789f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:26.545141 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13788c0-8967-41ed-b2b5-b048e1f789f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:26.545520 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13788c0-8967-41ed-b2b5-b048e1f789f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:26.546467 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13788c0-8967-41ed-b2b5-b048e1f789f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:26.549096 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a786f-b3b5-44db-8bae-2dfbd853c21d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:26.549297 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13788c0-8967-41ed-b2b5-b048e1f789f2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:26.558177 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d77c4-b869-45ba-acad-5fdd0c12b02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:26.559977 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d77c4-b869-45ba-acad-5fdd0c12b02b admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:26.560536 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d77c4-b869-45ba-acad-5fdd0c12b02b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:26.561034 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d77c4-b869-45ba-acad-5fdd0c12b02b admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:26.562453 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2d77c4-b869-45ba-acad-5fdd0c12b02b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:24:26.563032 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2d77c4-b869-45ba-acad-5fdd0c12b02b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:26.563844 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 780/1576] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:24:26.566420 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2d77c4-b869-45ba-acad-5fdd0c12b02b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:26.568363 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13788c0-8967-41ed-b2b5-b048e1f789f2 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:26.580812 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13788c0-8967-41ed-b2b5-b048e1f789f2 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:26.587049 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921e403-fabe-4a70-ab72-9bb0cec14126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:26.587787 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921e403-fabe-4a70-ab72-9bb0cec14126 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:26.588075 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921e403-fabe-4a70-ab72-9bb0cec14126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:26.588354 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921e403-fabe-4a70-ab72-9bb0cec14126 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:26.589243 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f921e403-fabe-4a70-ab72-9bb0cec14126 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:26.591780 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f921e403-fabe-4a70-ab72-9bb0cec14126 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:26.599640 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 797/1577] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:24:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:24:26.599640 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d77c4-b869-45ba-acad-5fdd0c12b02b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:26.612997 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d77c4-b869-45ba-acad-5fdd0c12b02b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:26.614088 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 781/1578] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:24:26.616474 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4689d2-606a-4369-95e9-a3c0e2eaf743 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:26.617002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4689d2-606a-4369-95e9-a3c0e2eaf743 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:26.617632 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4689d2-606a-4369-95e9-a3c0e2eaf743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:26.625987 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3149d-7913-4b22-a794-d424e6efc724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:26.627296 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3149d-7913-4b22-a794-d424e6efc724 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:26.627946 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3149d-7913-4b22-a794-d424e6efc724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:26.628549 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3149d-7913-4b22-a794-d424e6efc724 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:26.629923 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a3149d-7913-4b22-a794-d424e6efc724 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:26.633491 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a3149d-7913-4b22-a794-d424e6efc724 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:26.644175 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-9e4689d2-606a-4369-95e9-a3c0e2eaf743 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:26.645602 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9e4689d2-606a-4369-95e9-a3c0e2eaf743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:26.657411 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 782/1579] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:26.659271 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4689d2-606a-4369-95e9-a3c0e2eaf743 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:26.676165 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 798/1580] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:24:26 2022] POST /identity/v3/users => generated 287 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:24:26.678688 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3149d-7913-4b22-a794-d424e6efc724 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:26.681242 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4689d2-606a-4369-95e9-a3c0e2eaf743 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:26.689203 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc22033-2dec-41ad-9c16-609d53f3b27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:26.689961 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc22033-2dec-41ad-9c16-609d53f3b27d admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:26.690378 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc22033-2dec-41ad-9c16-609d53f3b27d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:26.690783 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc22033-2dec-41ad-9c16-609d53f3b27d admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:26.691835 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc22033-2dec-41ad-9c16-609d53f3b27d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:26.694413 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc22033-2dec-41ad-9c16-609d53f3b27d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:26.698979 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3149d-7913-4b22-a794-d424e6efc724 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:26.707038 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa3a22b-2ac0-487f-ac7f-3ad2302594f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:26.708289 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3a22b-2ac0-487f-ac7f-3ad2302594f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:26.708945 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3a22b-2ac0-487f-ac7f-3ad2302594f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:26.709398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3a22b-2ac0-487f-ac7f-3ad2302594f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:26.711257 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa3a22b-2ac0-487f-ac7f-3ad2302594f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:26.714685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa3a22b-2ac0-487f-ac7f-3ad2302594f4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:26.734383 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 783/1581] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:24:26 2022] POST /identity/v3/users => generated 297 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:24:26.734383 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc22033-2dec-41ad-9c16-609d53f3b27d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:26.750614 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc22033-2dec-41ad-9c16-609d53f3b27d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:26.756837 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 799/1582] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:24:26 2022] POST /identity/v3/users => generated 303 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:24:26.756837 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa3a22b-2ac0-487f-ac7f-3ad2302594f4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:26.764965 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe4214-6444-4f9e-a8e0-3c5367fef769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:26.766132 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe4214-6444-4f9e-a8e0-3c5367fef769 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:26.766614 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe4214-6444-4f9e-a8e0-3c5367fef769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:26.767187 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa3a22b-2ac0-487f-ac7f-3ad2302594f4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:26.767403 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe4214-6444-4f9e-a8e0-3c5367fef769 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:26.768689 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abe4214-6444-4f9e-a8e0-3c5367fef769 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:26.772515 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abe4214-6444-4f9e-a8e0-3c5367fef769 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:26.773216 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc100d-2b83-4636-8e2d-6a0ef3f0e35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:26.774307 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc100d-2b83-4636-8e2d-6a0ef3f0e35b admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:26.774749 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc100d-2b83-4636-8e2d-6a0ef3f0e35b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:26.775177 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc100d-2b83-4636-8e2d-6a0ef3f0e35b admin admin] PATH_INFO: `/v3/projects/8185baa0f3e3465f813c0e2dc29ad847/users/6ecf34ceee294b53b5be7c1723b42950/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:26.796615 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dc100d-2b83-4636-8e2d-6a0ef3f0e35b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8185baa0f3e3465f813c0e2dc29ad847, user_id=6ecf34ceee294b53b5be7c1723b42950, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:26.800529 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dc100d-2b83-4636-8e2d-6a0ef3f0e35b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:26.820593 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 800/1583] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:24:26 2022] PUT /identity/v3/projects/8185baa0f3e3465f813c0e2dc29ad847/users/6ecf34ceee294b53b5be7c1723b42950/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:24:26.821973 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc100d-2b83-4636-8e2d-6a0ef3f0e35b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:26.833181 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 784/1584] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:24:26 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:24:26.833181 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe4214-6444-4f9e-a8e0-3c5367fef769 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:26.904504 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:26.911171 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:26.916305 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe4214-6444-4f9e-a8e0-3c5367fef769 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:26.923660 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc100d-2b83-4636-8e2d-6a0ef3f0e35b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:26.923894 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a106a-8d80-491d-a23c-9d8c09b2f7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:26.924548 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a106a-8d80-491d-a23c-9d8c09b2f7f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:26.924915 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a106a-8d80-491d-a23c-9d8c09b2f7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:26.925383 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a106a-8d80-491d-a23c-9d8c09b2f7f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:26.926606 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a106a-8d80-491d-a23c-9d8c09b2f7f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:26.928413 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a106a-8d80-491d-a23c-9d8c09b2f7f8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:26.929677 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11f9529-6223-4bbe-80c5-ed240cd8ce8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:26.930365 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f9529-6223-4bbe-80c5-ed240cd8ce8f admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:26.930743 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f9529-6223-4bbe-80c5-ed240cd8ce8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:26.931558 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f9529-6223-4bbe-80c5-ed240cd8ce8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:26.932006 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11f9529-6223-4bbe-80c5-ed240cd8ce8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:26.934604 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11f9529-6223-4bbe-80c5-ed240cd8ce8f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:26.941628 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 785/1585] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:24:26.942498 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a106a-8d80-491d-a23c-9d8c09b2f7f8 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:26.975131 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 801/1586] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:24:26 2022] POST /identity/v3/projects => generated 365 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:24:26.976592 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11f9529-6223-4bbe-80c5-ed240cd8ce8f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:27.035188 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a106a-8d80-491d-a23c-9d8c09b2f7f8 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:27.082872 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11f9529-6223-4bbe-80c5-ed240cd8ce8f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:27.091700 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69e7ab-692b-48b0-8691-8505d788f8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:27.092818 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69e7ab-692b-48b0-8691-8505d788f8a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:27.093328 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69e7ab-692b-48b0-8691-8505d788f8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:27.093836 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69e7ab-692b-48b0-8691-8505d788f8a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:27.095220 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d69e7ab-692b-48b0-8691-8505d788f8a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:27.098457 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d69e7ab-692b-48b0-8691-8505d788f8a3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:27.113637 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446fc477-4d56-47bf-b3f1-e37644a58ded service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:27.114766 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446fc477-4d56-47bf-b3f1-e37644a58ded service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:27.115260 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446fc477-4d56-47bf-b3f1-e37644a58ded service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:27.115815 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446fc477-4d56-47bf-b3f1-e37644a58ded service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:27.123132 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 802/1587] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:24:27.123132 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69e7ab-692b-48b0-8691-8505d788f8a3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:27.137950 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69e7ab-692b-48b0-8691-8505d788f8a3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:27.145308 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d60660-9865-4661-917a-20ba326930b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:27.146702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d60660-9865-4661-917a-20ba326930b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:27.147199 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d60660-9865-4661-917a-20ba326930b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:27.147753 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d60660-9865-4661-917a-20ba326930b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:27.149177 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446fc477-4d56-47bf-b3f1-e37644a58ded service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:27.149380 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d60660-9865-4661-917a-20ba326930b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:27.152514 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d60660-9865-4661-917a-20ba326930b6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:27.152819 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446fc477-4d56-47bf-b3f1-e37644a58ded service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:27.163905 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 786/1588] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:24:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:24:27.164561 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446fc477-4d56-47bf-b3f1-e37644a58ded service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:27.177803 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 803/1589] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:24:27.179049 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446fc477-4d56-47bf-b3f1-e37644a58ded service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:27.179214 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a5cf6-3565-4e34-9729-7fd8734ecaee None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:27.179214 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a5cf6-3565-4e34-9729-7fd8734ecaee None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:27.179214 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a5cf6-3565-4e34-9729-7fd8734ecaee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:27.185397 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4218f0-1fdf-4e29-af47-529d89b6ab48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:27.186097 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4218f0-1fdf-4e29-af47-529d89b6ab48 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:27.186382 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4218f0-1fdf-4e29-af47-529d89b6ab48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:27.186666 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4218f0-1fdf-4e29-af47-529d89b6ab48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:27.187605 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4218f0-1fdf-4e29-af47-529d89b6ab48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:27.189273 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4218f0-1fdf-4e29-af47-529d89b6ab48 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:27.207122 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 787/1590] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:24:27.207122 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4218f0-1fdf-4e29-af47-529d89b6ab48 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:27.211854 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-602a5cf6-3565-4e34-9729-7fd8734ecaee None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:27.214132 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-602a5cf6-3565-4e34-9729-7fd8734ecaee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:27.224268 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4218f0-1fdf-4e29-af47-529d89b6ab48 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:27.229443 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a78c2d0-109a-414a-af93-770aa08431d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:27.230846 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78c2d0-109a-414a-af93-770aa08431d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:27.232396 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78c2d0-109a-414a-af93-770aa08431d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:27.234016 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 804/1591] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:27.235140 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a5cf6-3565-4e34-9729-7fd8734ecaee None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:27.235773 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78c2d0-109a-414a-af93-770aa08431d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:27.240030 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a78c2d0-109a-414a-af93-770aa08431d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:27.247992 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a78c2d0-109a-414a-af93-770aa08431d5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:27.249832 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a5cf6-3565-4e34-9729-7fd8734ecaee None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:27.257821 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acbc828-a09f-4a81-b8d2-21f4cccdace1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:27.259327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbc828-a09f-4a81-b8d2-21f4cccdace1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:27.259884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbc828-a09f-4a81-b8d2-21f4cccdace1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:27.260382 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbc828-a09f-4a81-b8d2-21f4cccdace1 admin admin] PATH_INFO: `/v3/projects/4869e3056eb340c7a6a45949b97a8e7c/users/5f5358220a97443083b84a05ddbbad3f/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:27.284500 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 788/1592] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:24:27 2022] POST /identity/v3/users => generated 290 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:24:27.285858 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a78c2d0-109a-414a-af93-770aa08431d5 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:27.301257 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acbc828-a09f-4a81-b8d2-21f4cccdace1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4869e3056eb340c7a6a45949b97a8e7c, user_id=5f5358220a97443083b84a05ddbbad3f, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:27.306096 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a78c2d0-109a-414a-af93-770aa08431d5 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:27.307628 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acbc828-a09f-4a81-b8d2-21f4cccdace1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:27.312494 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22453a7-b231-4d64-b27f-deef5b9af89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:27.313348 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22453a7-b231-4d64-b27f-deef5b9af89d admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:27.313605 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22453a7-b231-4d64-b27f-deef5b9af89d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:27.314335 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22453a7-b231-4d64-b27f-deef5b9af89d admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:27.315331 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22453a7-b231-4d64-b27f-deef5b9af89d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:27.319246 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22453a7-b231-4d64-b27f-deef5b9af89d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:27.334697 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 805/1593] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:24:27 2022] PUT /identity/v3/projects/4869e3056eb340c7a6a45949b97a8e7c/users/5f5358220a97443083b84a05ddbbad3f/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:24:27.336842 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acbc828-a09f-4a81-b8d2-21f4cccdace1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:27.386770 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 789/1594] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:24:27 2022] POST /identity/v3/users => generated 283 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:24:27.388337 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22453a7-b231-4d64-b27f-deef5b9af89d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:27.408254 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22453a7-b231-4d64-b27f-deef5b9af89d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:27.490056 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acbc828-a09f-4a81-b8d2-21f4cccdace1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:27.496934 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e9fae-0b1f-41c7-8cf3-81a4fc42dd00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:27.497885 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e9fae-0b1f-41c7-8cf3-81a4fc42dd00 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:27.498376 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e9fae-0b1f-41c7-8cf3-81a4fc42dd00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:27.498821 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e9fae-0b1f-41c7-8cf3-81a4fc42dd00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:27.499788 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e9fae-0b1f-41c7-8cf3-81a4fc42dd00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:27.501474 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e9fae-0b1f-41c7-8cf3-81a4fc42dd00 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:27.514528 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 806/1595] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:24:27.515427 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e9fae-0b1f-41c7-8cf3-81a4fc42dd00 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:27.520573 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe98e955-28a8-4ada-8188-2d8d4c578957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:27.521448 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe98e955-28a8-4ada-8188-2d8d4c578957 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:27.521843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe98e955-28a8-4ada-8188-2d8d4c578957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:27.522433 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe98e955-28a8-4ada-8188-2d8d4c578957 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:27.523267 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe98e955-28a8-4ada-8188-2d8d4c578957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:27.525945 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe98e955-28a8-4ada-8188-2d8d4c578957 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:27.529896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e9fae-0b1f-41c7-8cf3-81a4fc42dd00 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:27.535121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d180e-2641-483e-ad47-f545a79b9027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:27.536384 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d180e-2641-483e-ad47-f545a79b9027 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:27.536384 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d180e-2641-483e-ad47-f545a79b9027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:27.536384 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9d180e-2641-483e-ad47-f545a79b9027 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:27.537156 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9d180e-2641-483e-ad47-f545a79b9027 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:27.538811 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9d180e-2641-483e-ad47-f545a79b9027 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:27.557258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 807/1596] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:24:27.557258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d180e-2641-483e-ad47-f545a79b9027 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:27.557582 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 790/1597] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:24:27.557582 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe98e955-28a8-4ada-8188-2d8d4c578957 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:27.566087 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:27.577513 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe98e955-28a8-4ada-8188-2d8d4c578957 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:27.581694 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:27.584652 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c81603-84b9-4054-9fce-9bfd551c3a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:27.585373 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c81603-84b9-4054-9fce-9bfd551c3a98 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:27.585646 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c81603-84b9-4054-9fce-9bfd551c3a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:27.585931 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c81603-84b9-4054-9fce-9bfd551c3a98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:27.587098 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c81603-84b9-4054-9fce-9bfd551c3a98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:27.588709 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c81603-84b9-4054-9fce-9bfd551c3a98 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:27.604424 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 791/1598] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:24:27.611546 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a4ba2-24bf-4bad-8812-1f1df48fca4d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:27.612004 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a4ba2-24bf-4bad-8812-1f1df48fca4d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:27.612224 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a4ba2-24bf-4bad-8812-1f1df48fca4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:27.661946 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b88a4ba2-24bf-4bad-8812-1f1df48fca4d None None] MFA Rules not processed for user `5f5358220a97443083b84a05ddbbad3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:27.664064 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b88a4ba2-24bf-4bad-8812-1f1df48fca4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:27.670918 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9d180e-2641-483e-ad47-f545a79b9027 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:27.681414 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12bb771-7a74-47e5-9415-92ee57c66596 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:27.682893 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12bb771-7a74-47e5-9415-92ee57c66596 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:27.683591 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12bb771-7a74-47e5-9415-92ee57c66596 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:27.684907 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12bb771-7a74-47e5-9415-92ee57c66596 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:27.706511 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12bb771-7a74-47e5-9415-92ee57c66596 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:27.709601 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12bb771-7a74-47e5-9415-92ee57c66596 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:27.728694 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 808/1599] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:24:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:24:27.730785 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12bb771-7a74-47e5-9415-92ee57c66596 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:27.745265 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12bb771-7a74-47e5-9415-92ee57c66596 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:27.753696 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6b0b8-3d5c-417d-bfc9-fc616c00b99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:27.754918 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6b0b8-3d5c-417d-bfc9-fc616c00b99c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:27.755465 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6b0b8-3d5c-417d-bfc9-fc616c00b99c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:27.756130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6b0b8-3d5c-417d-bfc9-fc616c00b99c admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:27.757745 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6b0b8-3d5c-417d-bfc9-fc616c00b99c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:27.760962 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6b0b8-3d5c-417d-bfc9-fc616c00b99c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:27.773849 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 792/1600] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:27 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:27.775076 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88a4ba2-24bf-4bad-8812-1f1df48fca4d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:27.783473 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 809/1601] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:24:27.785006 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6b0b8-3d5c-417d-bfc9-fc616c00b99c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:27.793098 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88a4ba2-24bf-4bad-8812-1f1df48fca4d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:27.800643 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e48b3-e650-4815-9611-facc089ad98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:27.802332 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e48b3-e650-4815-9611-facc089ad98a admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:27.802952 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e48b3-e650-4815-9611-facc089ad98a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:27.803307 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e48b3-e650-4815-9611-facc089ad98a admin admin] PATH_INFO: `/v3/projects/2c772444cd3a43aba3efd52a54979d39/users/2e1f8b7984434ea1a0883feccec20d18/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:27.805772 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6b0b8-3d5c-417d-bfc9-fc616c00b99c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:27.816014 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf5cec-1c54-47e2-bc18-da3a1369472b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:27.816948 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf5cec-1c54-47e2-bc18-da3a1369472b admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:27.817299 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf5cec-1c54-47e2-bc18-da3a1369472b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:27.817580 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf5cec-1c54-47e2-bc18-da3a1369472b admin admin] PATH_INFO: `/v3/projects/1df108d610134859919e6bf8903f6190/users/0ed3ed6183d34c73ba40d13af037f838/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:27.833159 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4e48b3-e650-4815-9611-facc089ad98a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c772444cd3a43aba3efd52a54979d39, user_id=2e1f8b7984434ea1a0883feccec20d18, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:27.836414 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4e48b3-e650-4815-9611-facc089ad98a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:27.842785 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedf5cec-1c54-47e2-bc18-da3a1369472b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1df108d610134859919e6bf8903f6190, user_id=0ed3ed6183d34c73ba40d13af037f838, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:27.846404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedf5cec-1c54-47e2-bc18-da3a1369472b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:27.856063 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 793/1602] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:24:27 2022] PUT /identity/v3/projects/2c772444cd3a43aba3efd52a54979d39/users/2e1f8b7984434ea1a0883feccec20d18/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:24:27.865920 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e48b3-e650-4815-9611-facc089ad98a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:27.868122 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 810/1603] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:24:27 2022] PUT /identity/v3/projects/1df108d610134859919e6bf8903f6190/users/0ed3ed6183d34c73ba40d13af037f838/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:24:27.869286 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf5cec-1c54-47e2-bc18-da3a1369472b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:28.049793 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e48b3-e650-4815-9611-facc089ad98a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:28.082886 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf5cec-1c54-47e2-bc18-da3a1369472b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:28.110837 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07acd7ae-54a6-4c71-a98e-ae19a3e11222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:28.111992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07acd7ae-54a6-4c71-a98e-ae19a3e11222 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:28.112594 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07acd7ae-54a6-4c71-a98e-ae19a3e11222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:28.113049 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07acd7ae-54a6-4c71-a98e-ae19a3e11222 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:28.114501 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07acd7ae-54a6-4c71-a98e-ae19a3e11222 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:28.117559 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07acd7ae-54a6-4c71-a98e-ae19a3e11222 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:28.148118 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 811/1604] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:24:28.149214 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07acd7ae-54a6-4c71-a98e-ae19a3e11222 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:28.175984 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07acd7ae-54a6-4c71-a98e-ae19a3e11222 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:28.184794 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5051831-7d98-4623-b9b5-f02acbf2e2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:28.185825 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5051831-7d98-4623-b9b5-f02acbf2e2c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:28.186122 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5051831-7d98-4623-b9b5-f02acbf2e2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:28.186523 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5051831-7d98-4623-b9b5-f02acbf2e2c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:28.187363 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5051831-7d98-4623-b9b5-f02acbf2e2c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:28.189506 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5051831-7d98-4623-b9b5-f02acbf2e2c3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:28.194511 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934b735-37ef-4fdb-b9c8-abe7cb0acf35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:28.197421 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934b735-37ef-4fdb-b9c8-abe7cb0acf35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:28.198201 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934b735-37ef-4fdb-b9c8-abe7cb0acf35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:28.198964 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934b735-37ef-4fdb-b9c8-abe7cb0acf35 admin admin] PATH_INFO: `/v3/projects/32f923f138fb4564b7fd28431803fc3b/users/d6030ab8afa3404f91b6a2bff8754964/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:28.204889 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 812/1605] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:24:28 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:24:28.206801 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5051831-7d98-4623-b9b5-f02acbf2e2c3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:28.226790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:28.244100 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9934b735-37ef-4fdb-b9c8-abe7cb0acf35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32f923f138fb4564b7fd28431803fc3b, user_id=d6030ab8afa3404f91b6a2bff8754964, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:28.249958 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9934b735-37ef-4fdb-b9c8-abe7cb0acf35 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:28.281992 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 794/1606] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:24:27 2022] PUT /identity/v3/projects/32f923f138fb4564b7fd28431803fc3b/users/d6030ab8afa3404f91b6a2bff8754964/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:24:28.296967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006167cb-78e6-4ce5-8dc4-56ee101abf13 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:28.296967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006167cb-78e6-4ce5-8dc4-56ee101abf13 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:28.296967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006167cb-78e6-4ce5-8dc4-56ee101abf13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:28.332349 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-006167cb-78e6-4ce5-8dc4-56ee101abf13 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:28.335533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-006167cb-78e6-4ce5-8dc4-56ee101abf13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:28.336527 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5051831-7d98-4623-b9b5-f02acbf2e2c3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:28.390433 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:28.479630 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288fa9f2-4045-4ee7-b617-b421f4eb4fdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:28.481877 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288fa9f2-4045-4ee7-b617-b421f4eb4fdd service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:28.482432 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288fa9f2-4045-4ee7-b617-b421f4eb4fdd service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:28.483331 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288fa9f2-4045-4ee7-b617-b421f4eb4fdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:28.506748 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 795/1607] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:28.509478 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8254baf1-6e8f-4ad2-ab9b-21d1166921a5 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:28.510040 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8254baf1-6e8f-4ad2-ab9b-21d1166921a5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:28.510478 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8254baf1-6e8f-4ad2-ab9b-21d1166921a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:28.565379 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8254baf1-6e8f-4ad2-ab9b-21d1166921a5 None None] MFA Rules not processed for user `2e1f8b7984434ea1a0883feccec20d18`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:28.567586 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8254baf1-6e8f-4ad2-ab9b-21d1166921a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:28.602154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288fa9f2-4045-4ee7-b617-b421f4eb4fdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:28.604056 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288fa9f2-4045-4ee7-b617-b421f4eb4fdd service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:28.617822 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 813/1608] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:24:28 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:24:28.618319 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288fa9f2-4045-4ee7-b617-b421f4eb4fdd service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:28.638268 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288fa9f2-4045-4ee7-b617-b421f4eb4fdd service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:28.653538 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc071a-0cbf-409e-b655-eaa0070d87a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:28.658796 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc071a-0cbf-409e-b655-eaa0070d87a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:28.659175 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc071a-0cbf-409e-b655-eaa0070d87a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:28.659616 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc071a-0cbf-409e-b655-eaa0070d87a5 admin admin] PATH_INFO: `/v3/projects/a6744feae17b4b32a60e06d373feb52d/users/30efc9bc66ce498d9d94980d85152694/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:28.696291 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 796/1609] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:28.698468 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bc071a-0cbf-409e-b655-eaa0070d87a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6744feae17b4b32a60e06d373feb52d, user_id=30efc9bc66ce498d9d94980d85152694, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:28.700388 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8254baf1-6e8f-4ad2-ab9b-21d1166921a5 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:28.704786 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bc071a-0cbf-409e-b655-eaa0070d87a5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:28.725655 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8254baf1-6e8f-4ad2-ab9b-21d1166921a5 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:28.738456 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529695f-332f-4a6f-a491-94f5aee78b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:28.740705 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529695f-332f-4a6f-a491-94f5aee78b7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:28.742017 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529695f-332f-4a6f-a491-94f5aee78b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:28.743326 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529695f-332f-4a6f-a491-94f5aee78b7a admin admin] PATH_INFO: `/v3/projects/70b28a4870d94263bf7a134130078181/users/6f974d50b4ec41ddbf00e548606f2ba1/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:28.753175 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 814/1610] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:24:28 2022] PUT /identity/v3/projects/a6744feae17b4b32a60e06d373feb52d/users/30efc9bc66ce498d9d94980d85152694/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:24:28.753175 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8fc9f-8cc2-46e7-83db-778080a9b21e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:28.753175 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8fc9f-8cc2-46e7-83db-778080a9b21e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:28.753175 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8fc9f-8cc2-46e7-83db-778080a9b21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:28.780902 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529695f-332f-4a6f-a491-94f5aee78b7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=70b28a4870d94263bf7a134130078181, user_id=6f974d50b4ec41ddbf00e548606f2ba1, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:28.784696 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529695f-332f-4a6f-a491-94f5aee78b7a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:28.847597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 797/1611] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:24:28 2022] PUT /identity/v3/projects/70b28a4870d94263bf7a134130078181/users/6f974d50b4ec41ddbf00e548606f2ba1/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:24:28.847597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529695f-332f-4a6f-a491-94f5aee78b7a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:28.854156 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b6a8fc9f-8cc2-46e7-83db-778080a9b21e None None] MFA Rules not processed for user `d6030ab8afa3404f91b6a2bff8754964`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:28.859897 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b6a8fc9f-8cc2-46e7-83db-778080a9b21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:28.991879 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529695f-332f-4a6f-a491-94f5aee78b7a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:29.000097 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90407d8-d209-47b0-9d3f-c2718c6faf09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:29.001075 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90407d8-d209-47b0-9d3f-c2718c6faf09 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:29.001461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90407d8-d209-47b0-9d3f-c2718c6faf09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:29.001835 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90407d8-d209-47b0-9d3f-c2718c6faf09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:29.013634 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:29.075721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 815/1612] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:28 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:29.075721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197f0ab-da6f-4146-b40e-554952a0d7fb None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:29.075721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197f0ab-da6f-4146-b40e-554952a0d7fb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:29.075721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197f0ab-da6f-4146-b40e-554952a0d7fb None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:29.091739 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 816/1613] 10.222.0.18 () {54 vars in 962 bytes} [Mon Jul 25 19:24:29 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:24:29.097744 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5cd0f3-d720-42f1-b0b3-19a267e57abd None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:29.098631 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5cd0f3-d720-42f1-b0b3-19a267e57abd None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:29.099600 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5cd0f3-d720-42f1-b0b3-19a267e57abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:29.165651 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-bf5cd0f3-d720-42f1-b0b3-19a267e57abd None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:29.170071 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bf5cd0f3-d720-42f1-b0b3-19a267e57abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:29.180619 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90407d8-d209-47b0-9d3f-c2718c6faf09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:29.184051 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90407d8-d209-47b0-9d3f-c2718c6faf09 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:29.201468 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 817/1614] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:29.204266 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7f183-da2e-4a4b-9d68-8d942dcceab5 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:29.204761 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7f183-da2e-4a4b-9d68-8d942dcceab5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:29.205242 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf7f183-da2e-4a4b-9d68-8d942dcceab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:29.209689 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 798/1615] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:24:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:24:29.211487 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90407d8-d209-47b0-9d3f-c2718c6faf09 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:29.240612 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90407d8-d209-47b0-9d3f-c2718c6faf09 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:29.257316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3176ec-a0a6-4999-b337-9fa9792d2b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:29.259318 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3176ec-a0a6-4999-b337-9fa9792d2b33 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:29.259835 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3176ec-a0a6-4999-b337-9fa9792d2b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:29.262784 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3176ec-a0a6-4999-b337-9fa9792d2b33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:29.265203 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3176ec-a0a6-4999-b337-9fa9792d2b33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:29.270088 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3176ec-a0a6-4999-b337-9fa9792d2b33 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:29.278109 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2bf7f183-da2e-4a4b-9d68-8d942dcceab5 None None] MFA Rules not processed for user `6f974d50b4ec41ddbf00e548606f2ba1`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:29.279996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2bf7f183-da2e-4a4b-9d68-8d942dcceab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:29.325013 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 799/1616] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:24:29 2022] POST /identity/v3/projects => generated 405 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:24:29.326797 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e232c69-bdd2-4765-9047-56990fe6e820 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:29.327102 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e232c69-bdd2-4765-9047-56990fe6e820 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:29.327413 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e232c69-bdd2-4765-9047-56990fe6e820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:29.358917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:29.416072 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7e232c69-bdd2-4765-9047-56990fe6e820 None None] MFA Rules not processed for user `860e7d26ad364795bb25836aadd4ac7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:29.419285 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7e232c69-bdd2-4765-9047-56990fe6e820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:29.479762 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 818/1617] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:29 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:29.486114 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7f183-da2e-4a4b-9d68-8d942dcceab5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:29.526170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf7f183-da2e-4a4b-9d68-8d942dcceab5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:29.595483 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 800/1618] 10.222.0.18 () {58 vars in 1059 bytes} [Mon Jul 25 19:24:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:29.598576 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e232c69-bdd2-4765-9047-56990fe6e820 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:29.783855 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e232c69-bdd2-4765-9047-56990fe6e820 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:29.797718 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cef555-40bc-48de-94ec-bb6e0b20b26b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:29.798932 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cef555-40bc-48de-94ec-bb6e0b20b26b service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:29.799452 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cef555-40bc-48de-94ec-bb6e0b20b26b service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:29.799992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cef555-40bc-48de-94ec-bb6e0b20b26b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:29.807563 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af9ec42-c4ee-4ac4-b5d0-c4ec6b410b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:29.809273 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af9ec42-c4ee-4ac4-b5d0-c4ec6b410b96 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:29.809634 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af9ec42-c4ee-4ac4-b5d0-c4ec6b410b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:29.809923 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af9ec42-c4ee-4ac4-b5d0-c4ec6b410b96 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:29.810830 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af9ec42-c4ee-4ac4-b5d0-c4ec6b410b96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:29.837616 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af9ec42-c4ee-4ac4-b5d0-c4ec6b410b96 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:29.840596 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cef555-40bc-48de-94ec-bb6e0b20b26b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:29.843159 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cef555-40bc-48de-94ec-bb6e0b20b26b service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:29.852852 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:29.886569 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 801/1619] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:24:29 2022] POST /identity/v3/users => generated 309 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:24:29.887795 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af9ec42-c4ee-4ac4-b5d0-c4ec6b410b96 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:29.907164 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 819/1620] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:24:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:24:29.908738 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cef555-40bc-48de-94ec-bb6e0b20b26b service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:29.947890 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af9ec42-c4ee-4ac4-b5d0-c4ec6b410b96 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:29.950020 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cef555-40bc-48de-94ec-bb6e0b20b26b service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:29.961239 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240807ce-f9f2-46ac-827c-cd38438f1c6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:29.962042 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b98609-7c98-4527-a26a-e777cbda4651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:29.962421 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240807ce-f9f2-46ac-827c-cd38438f1c6f service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:29.962908 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240807ce-f9f2-46ac-827c-cd38438f1c6f service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:29.963548 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240807ce-f9f2-46ac-827c-cd38438f1c6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:29.968411 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b98609-7c98-4527-a26a-e777cbda4651 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:29.969080 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b98609-7c98-4527-a26a-e777cbda4651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:29.971620 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b98609-7c98-4527-a26a-e777cbda4651 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:29.975230 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b98609-7c98-4527-a26a-e777cbda4651 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:29.984781 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b98609-7c98-4527-a26a-e777cbda4651 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:30.152306 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240807ce-f9f2-46ac-827c-cd38438f1c6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:30.152306 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240807ce-f9f2-46ac-827c-cd38438f1c6f service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:30.152306 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 820/1621] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:24:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:24:30.153513 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240807ce-f9f2-46ac-827c-cd38438f1c6f service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:30.172262 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 802/1622] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:24:29 2022] POST /identity/v3/projects => generated 363 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:24:30.174335 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b98609-7c98-4527-a26a-e777cbda4651 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:30.196154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240807ce-f9f2-46ac-827c-cd38438f1c6f service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:30.505784 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b98609-7c98-4527-a26a-e777cbda4651 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:30.527587 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159fa99-9a5c-4bfa-8eae-868642a0e973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:30.532099 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159fa99-9a5c-4bfa-8eae-868642a0e973 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:30.534565 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159fa99-9a5c-4bfa-8eae-868642a0e973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:30.535281 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159fa99-9a5c-4bfa-8eae-868642a0e973 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:30.535802 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d74c35-b32a-4e25-9577-06d0618a5b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:30.537340 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c159fa99-9a5c-4bfa-8eae-868642a0e973 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:30.539678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d74c35-b32a-4e25-9577-06d0618a5b71 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:30.541876 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d74c35-b32a-4e25-9577-06d0618a5b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:30.543020 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d74c35-b32a-4e25-9577-06d0618a5b71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:30.549146 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d74c35-b32a-4e25-9577-06d0618a5b71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:30.552743 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c159fa99-9a5c-4bfa-8eae-868642a0e973 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:30.557539 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d74c35-b32a-4e25-9577-06d0618a5b71 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:30.615090 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 821/1623] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:24:30 2022] GET /identity/v3/roles => generated 1420 bytes in 464 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:24:30.622472 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d74c35-b32a-4e25-9577-06d0618a5b71 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:30.643434 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:30.664161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 803/1624] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:24:30 2022] POST /identity/v3/users => generated 288 bytes in 491 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:24:30.668486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159fa99-9a5c-4bfa-8eae-868642a0e973 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:30.674850 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d74c35-b32a-4e25-9577-06d0618a5b71 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:30.703851 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507916da-4515-4a41-9bcb-fb36b822ca5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:30.710395 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507916da-4515-4a41-9bcb-fb36b822ca5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:30.711307 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507916da-4515-4a41-9bcb-fb36b822ca5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:30.713903 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507916da-4515-4a41-9bcb-fb36b822ca5b admin admin] PATH_INFO: `/v3/projects/833d164d4c6d4f6697dfb6dcbf88b4a6/users/0bbd03f51dac47ddb5f61a522ba7fb8a/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:30.735289 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159fa99-9a5c-4bfa-8eae-868642a0e973 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:30.762583 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c000df-54f3-40b2-89ca-1c77a6a6bb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:30.767244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c000df-54f3-40b2-89ca-1c77a6a6bb29 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:30.770525 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c000df-54f3-40b2-89ca-1c77a6a6bb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:30.773879 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c000df-54f3-40b2-89ca-1c77a6a6bb29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:30.778113 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c000df-54f3-40b2-89ca-1c77a6a6bb29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:30.788607 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507916da-4515-4a41-9bcb-fb36b822ca5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=833d164d4c6d4f6697dfb6dcbf88b4a6, user_id=0bbd03f51dac47ddb5f61a522ba7fb8a, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:30.788866 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c000df-54f3-40b2-89ca-1c77a6a6bb29 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:30.802635 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507916da-4515-4a41-9bcb-fb36b822ca5b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:30.852890 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 804/1625] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:24:30 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:24:30.854571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c000df-54f3-40b2-89ca-1c77a6a6bb29 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:30.862851 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 822/1626] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:24:30 2022] PUT /identity/v3/projects/833d164d4c6d4f6697dfb6dcbf88b4a6/users/0bbd03f51dac47ddb5f61a522ba7fb8a/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:24:30.869159 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507916da-4515-4a41-9bcb-fb36b822ca5b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:30.909534 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c000df-54f3-40b2-89ca-1c77a6a6bb29 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:31.032961 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507916da-4515-4a41-9bcb-fb36b822ca5b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:31.062404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994c765-d885-49c0-8f22-36a1b19e2bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:31.075399 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994c765-d885-49c0-8f22-36a1b19e2bd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:31.076772 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994c765-d885-49c0-8f22-36a1b19e2bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:31.077661 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994c765-d885-49c0-8f22-36a1b19e2bd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:31.080226 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8994c765-d885-49c0-8f22-36a1b19e2bd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:31.088828 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8994c765-d885-49c0-8f22-36a1b19e2bd6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:31.153044 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 823/1627] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:24:30 2022] GET /identity/v3/roles => generated 1420 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:24:31.155063 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994c765-d885-49c0-8f22-36a1b19e2bd6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:31.180224 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeff984-f957-41b8-aa06-b123985a41e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:31.181366 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeff984-f957-41b8-aa06-b123985a41e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:31.181660 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeff984-f957-41b8-aa06-b123985a41e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:31.182289 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeff984-f957-41b8-aa06-b123985a41e5 admin admin] PATH_INFO: `/v3/projects/efbdd83c95884ccda10a6c0b43d8c09c/users/447def5b8e6f40bab08e0459b03ec9b3/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:31.199433 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994c765-d885-49c0-8f22-36a1b19e2bd6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:31.202305 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeff984-f957-41b8-aa06-b123985a41e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efbdd83c95884ccda10a6c0b43d8c09c, user_id=447def5b8e6f40bab08e0459b03ec9b3, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:31.205384 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeff984-f957-41b8-aa06-b123985a41e5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:31.217908 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ed7c8-8476-42c5-941f-45ff10e8b3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:31.218956 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ed7c8-8476-42c5-941f-45ff10e8b3a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:31.219256 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ed7c8-8476-42c5-941f-45ff10e8b3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:31.219668 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ed7c8-8476-42c5-941f-45ff10e8b3a7 admin admin] PATH_INFO: `/v3/projects/833d164d4c6d4f6697dfb6dcbf88b4a6/users/0bbd03f51dac47ddb5f61a522ba7fb8a/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:31.222502 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6ed7c8-8476-42c5-941f-45ff10e8b3a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=833d164d4c6d4f6697dfb6dcbf88b4a6, user_id=0bbd03f51dac47ddb5f61a522ba7fb8a, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:31.226089 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6ed7c8-8476-42c5-941f-45ff10e8b3a7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:31.242846 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 805/1629] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:24:30 2022] PUT /identity/v3/projects/efbdd83c95884ccda10a6c0b43d8c09c/users/447def5b8e6f40bab08e0459b03ec9b3/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:24:31.246878 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1135d-9f81-434f-b7ca-831b99fe8a2a None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:31.247451 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1135d-9f81-434f-b7ca-831b99fe8a2a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:31.247942 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1135d-9f81-434f-b7ca-831b99fe8a2a None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:31.250540 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 824/1629] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:24:31 2022] PUT /identity/v3/projects/833d164d4c6d4f6697dfb6dcbf88b4a6/users/0bbd03f51dac47ddb5f61a522ba7fb8a/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:24:31.251625 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ed7c8-8476-42c5-941f-45ff10e8b3a7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:31.265501 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 806/1630] 10.222.0.18 () {54 vars in 912 bytes} [Mon Jul 25 19:24:31 2022] GET /identity => generated 270 bytes in 22 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:24:31.267231 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8c536-4a81-49ca-afbf-271a91c6a836 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:31.269226 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8c536-4a81-49ca-afbf-271a91c6a836 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:31.269226 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f8c536-4a81-49ca-afbf-271a91c6a836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:31.316389 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-80f8c536-4a81-49ca-afbf-271a91c6a836 None None] MFA Rules not processed for user `0bbd03f51dac47ddb5f61a522ba7fb8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:31.318099 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-80f8c536-4a81-49ca-afbf-271a91c6a836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:31.363416 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:31.368423 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ed7c8-8476-42c5-941f-45ff10e8b3a7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:31.379128 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b0ba7-afef-4c6b-872a-fb9932497710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:31.379917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b0ba7-afef-4c6b-872a-fb9932497710 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:31.380862 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b0ba7-afef-4c6b-872a-fb9932497710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:31.381213 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b0ba7-afef-4c6b-872a-fb9932497710 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:31.382011 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455b0ba7-afef-4c6b-872a-fb9932497710 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:31.385191 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455b0ba7-afef-4c6b-872a-fb9932497710 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:31.401349 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 825/1631] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:24:31 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:24:31.402341 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26197313-f756-4482-a3e6-d4e36fed7de0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:31.402714 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26197313-f756-4482-a3e6-d4e36fed7de0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:31.404140 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26197313-f756-4482-a3e6-d4e36fed7de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:31.436724 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-26197313-f756-4482-a3e6-d4e36fed7de0 None None] MFA Rules not processed for user `860e7d26ad364795bb25836aadd4ac7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:31.442010 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-26197313-f756-4482-a3e6-d4e36fed7de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:31.505334 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 807/1632] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:31 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:31.509672 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f8c536-4a81-49ca-afbf-271a91c6a836 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:31.550907 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f8c536-4a81-49ca-afbf-271a91c6a836 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:31.563690 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579df6fe-154f-40dc-b90d-9b71df23010e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:31.564789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579df6fe-154f-40dc-b90d-9b71df23010e admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:31.565231 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579df6fe-154f-40dc-b90d-9b71df23010e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:31.565684 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579df6fe-154f-40dc-b90d-9b71df23010e admin admin] PATH_INFO: `/v3/projects/efbdd83c95884ccda10a6c0b43d8c09c/users/447def5b8e6f40bab08e0459b03ec9b3/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:31.569072 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:31.570993 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579df6fe-154f-40dc-b90d-9b71df23010e admin admin] RBAC: Authorizing `identity:create_grant(project_id=efbdd83c95884ccda10a6c0b43d8c09c, user_id=447def5b8e6f40bab08e0459b03ec9b3, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:31.576329 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579df6fe-154f-40dc-b90d-9b71df23010e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:31.595382 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 808/1633] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:24:31 2022] PUT /identity/v3/projects/efbdd83c95884ccda10a6c0b43d8c09c/users/447def5b8e6f40bab08e0459b03ec9b3/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:24:31.596463 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579df6fe-154f-40dc-b90d-9b71df23010e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:31.629933 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 826/1634] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Jul 25 19:24:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:31.639209 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad4caf-7609-42cc-b6c9-510e09806f6f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:31.639653 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad4caf-7609-42cc-b6c9-510e09806f6f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:31.640051 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad4caf-7609-42cc-b6c9-510e09806f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:31.751859 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-29ad4caf-7609-42cc-b6c9-510e09806f6f None None] MFA Rules not processed for user `447def5b8e6f40bab08e0459b03ec9b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:31.753515 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-29ad4caf-7609-42cc-b6c9-510e09806f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:31.791480 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579df6fe-154f-40dc-b90d-9b71df23010e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:31.803375 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a322acb-00a7-4a9f-a600-b0562a854548 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:31.804126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a322acb-00a7-4a9f-a600-b0562a854548 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:31.804356 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a322acb-00a7-4a9f-a600-b0562a854548 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:31.804633 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a322acb-00a7-4a9f-a600-b0562a854548 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:31.936685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 827/1635] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:31 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:31.938933 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad4caf-7609-42cc-b6c9-510e09806f6f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:31.976682 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad4caf-7609-42cc-b6c9-510e09806f6f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:31.981195 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a322acb-00a7-4a9f-a600-b0562a854548 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:31.985890 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a322acb-00a7-4a9f-a600-b0562a854548 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:31.994634 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f2edaa-2c0a-4a1f-a139-2452c693ef48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:31.996363 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2edaa-2c0a-4a1f-a139-2452c693ef48 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:31.996928 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2edaa-2c0a-4a1f-a139-2452c693ef48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:31.997209 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2edaa-2c0a-4a1f-a139-2452c693ef48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:32.011905 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 809/1636] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:24:31 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:24:32.164141 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a322acb-00a7-4a9f-a600-b0562a854548 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:32.187004 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a322acb-00a7-4a9f-a600-b0562a854548 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:32.206016 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bff8899-3a50-4495-88ff-5619cdc6b3f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:32.207084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff8899-3a50-4495-88ff-5619cdc6b3f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:32.207498 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff8899-3a50-4495-88ff-5619cdc6b3f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:32.208152 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff8899-3a50-4495-88ff-5619cdc6b3f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:32.228310 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bff8899-3a50-4495-88ff-5619cdc6b3f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:32.231820 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bff8899-3a50-4495-88ff-5619cdc6b3f4 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:32.259258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f2edaa-2c0a-4a1f-a139-2452c693ef48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:32.260840 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f2edaa-2c0a-4a1f-a139-2452c693ef48 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:32.282879 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 810/1637] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:24:32 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:24:32.302147 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 828/1638] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:24:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:24:32.505599 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84e952-d67e-40e0-89e8-b410776f7f71 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:32.506148 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84e952-d67e-40e0-89e8-b410776f7f71 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:32.506759 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84e952-d67e-40e0-89e8-b410776f7f71 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:32.517539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 811/1639] 10.222.0.18 () {54 vars in 962 bytes} [Mon Jul 25 19:24:32 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:24:32.537156 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6867e62-3985-4ee9-8a2e-bcf46aa2d8c8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:32.538587 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6867e62-3985-4ee9-8a2e-bcf46aa2d8c8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:32.538889 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6867e62-3985-4ee9-8a2e-bcf46aa2d8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:32.627504 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e6867e62-3985-4ee9-8a2e-bcf46aa2d8c8 None None] MFA Rules not processed for user `860e7d26ad364795bb25836aadd4ac7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:32.630166 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e6867e62-3985-4ee9-8a2e-bcf46aa2d8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:32.673281 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 829/1640] 10.222.0.18 () {58 vars in 1059 bytes} [Mon Jul 25 19:24:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:32.678829 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef5d4ca-05f5-4633-ac2c-fbad37019c89 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:32.680273 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef5d4ca-05f5-4633-ac2c-fbad37019c89 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:32.680273 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef5d4ca-05f5-4633-ac2c-fbad37019c89 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:32.683705 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 812/1641] 10.222.0.18 () {54 vars in 912 bytes} [Mon Jul 25 19:24:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:24:32.693219 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074d33d-b961-4aae-b545-3b370be28034 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:32.704191 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074d33d-b961-4aae-b545-3b370be28034 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:32.704191 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074d33d-b961-4aae-b545-3b370be28034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:32.745785 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a074d33d-b961-4aae-b545-3b370be28034 None None] MFA Rules not processed for user `860e7d26ad364795bb25836aadd4ac7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:32.752369 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a074d33d-b961-4aae-b545-3b370be28034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:32.796175 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef5d4ca-05f5-4633-ac2c-fbad37019c89 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:32.820883 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 830/1642] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Jul 25 19:24:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:32.856499 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef5d4ca-05f5-4633-ac2c-fbad37019c89 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:32.867147 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0d7b3e-58d0-40cb-9700-44e6fead6d5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:32.871670 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0d7b3e-58d0-40cb-9700-44e6fead6d5e service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:32.876322 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0d7b3e-58d0-40cb-9700-44e6fead6d5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:32.879588 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0d7b3e-58d0-40cb-9700-44e6fead6d5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:33.118445 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:33.225876 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0d7b3e-58d0-40cb-9700-44e6fead6d5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:33.228910 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0d7b3e-58d0-40cb-9700-44e6fead6d5e service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:33.281752 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 813/1643] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:24:32 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:24:33.446453 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a074d33d-b961-4aae-b545-3b370be28034 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:33.456964 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0d7b3e-58d0-40cb-9700-44e6fead6d5e service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:33.476347 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0d7b3e-58d0-40cb-9700-44e6fead6d5e service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:33.484925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bcb0a4-2bd3-4d39-be4a-ad1957d94d8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:33.486397 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcb0a4-2bd3-4d39-be4a-ad1957d94d8c service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:33.486905 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcb0a4-2bd3-4d39-be4a-ad1957d94d8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:33.487714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcb0a4-2bd3-4d39-be4a-ad1957d94d8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:33.546758 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bcb0a4-2bd3-4d39-be4a-ad1957d94d8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:33.554351 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bcb0a4-2bd3-4d39-be4a-ad1957d94d8c service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:33.634426 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 814/1644] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:24:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:24:33.753929 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:33.801751 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a074d33d-b961-4aae-b545-3b370be28034 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:33.818968 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3017f5d4-a333-4ac1-967b-9c9b5b25ff7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:33.820197 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017f5d4-a333-4ac1-967b-9c9b5b25ff7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:33.820620 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017f5d4-a333-4ac1-967b-9c9b5b25ff7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:33.821004 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017f5d4-a333-4ac1-967b-9c9b5b25ff7b admin admin] PATH_INFO: `/v3/users/d6030ab8afa3404f91b6a2bff8754964` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:33.827629 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3017f5d4-a333-4ac1-967b-9c9b5b25ff7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6030ab8afa3404f91b6a2bff8754964)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:33.829843 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3017f5d4-a333-4ac1-967b-9c9b5b25ff7b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:34.149827 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3017f5d4-a333-4ac1-967b-9c9b5b25ff7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6030ab8afa3404f91b6a2bff8754964'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:34.179031 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3017f5d4-a333-4ac1-967b-9c9b5b25ff7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6030ab8afa3404f91b6a2bff8754964'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:34.206255 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3017f5d4-a333-4ac1-967b-9c9b5b25ff7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6030ab8afa3404f91b6a2bff8754964'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:34.208397 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3017f5d4-a333-4ac1-967b-9c9b5b25ff7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6030ab8afa3404f91b6a2bff8754964'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:34.210286 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3017f5d4-a333-4ac1-967b-9c9b5b25ff7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6030ab8afa3404f91b6a2bff8754964'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:34.296632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 831/1645] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:24:33 2022] DELETE /identity/v3/users/d6030ab8afa3404f91b6a2bff8754964 => generated 0 bytes in 851 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:24:34.302394 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3038201-b8a0-4211-8c11-d70342d37fcf None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:34.303176 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3038201-b8a0-4211-8c11-d70342d37fcf None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:34.304823 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3038201-b8a0-4211-8c11-d70342d37fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:34.425549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f3038201-b8a0-4211-8c11-d70342d37fcf None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:34.429722 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f3038201-b8a0-4211-8c11-d70342d37fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:34.693815 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 815/1646] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:34.829705 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3017f5d4-a333-4ac1-967b-9c9b5b25ff7b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:34.832301 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3017f5d4-a333-4ac1-967b-9c9b5b25ff7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:35.128243 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3017f5d4-a333-4ac1-967b-9c9b5b25ff7b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:35.136393 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb9673f-97c3-4010-8d68-1a080aae4dbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:35.138174 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb9673f-97c3-4010-8d68-1a080aae4dbb service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:35.138922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb9673f-97c3-4010-8d68-1a080aae4dbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:35.139680 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb9673f-97c3-4010-8d68-1a080aae4dbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:35.176857 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb9673f-97c3-4010-8d68-1a080aae4dbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:35.179578 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb9673f-97c3-4010-8d68-1a080aae4dbb service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:35.213843 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 832/1647] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:24:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:24:36.804489 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3038201-b8a0-4211-8c11-d70342d37fcf None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:36.804489 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f3038201-b8a0-4211-8c11-d70342d37fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:36.822583 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3038201-b8a0-4211-8c11-d70342d37fcf None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:36.828800 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab2902-6cf9-4774-a3f5-39ffeeffe03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:36.830322 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab2902-6cf9-4774-a3f5-39ffeeffe03a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:36.830600 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab2902-6cf9-4774-a3f5-39ffeeffe03a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:36.831047 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab2902-6cf9-4774-a3f5-39ffeeffe03a admin admin] PATH_INFO: `/v3/users/2e1f8b7984434ea1a0883feccec20d18` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:36.832744 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ab2902-6cf9-4774-a3f5-39ffeeffe03a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e1f8b7984434ea1a0883feccec20d18)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:36.834478 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: WARNING py.warnings [None req-28ab2902-6cf9-4774-a3f5-39ffeeffe03a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 19:24:36.834478 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: warnings.warn(msg) Jul 25 19:24:36.834478 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: Jul 25 19:24:36.836674 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ab2902-6cf9-4774-a3f5-39ffeeffe03a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:36.989723 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-28ab2902-6cf9-4774-a3f5-39ffeeffe03a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e1f8b7984434ea1a0883feccec20d18'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:37.007150 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-28ab2902-6cf9-4774-a3f5-39ffeeffe03a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e1f8b7984434ea1a0883feccec20d18'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:37.046779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-28ab2902-6cf9-4774-a3f5-39ffeeffe03a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e1f8b7984434ea1a0883feccec20d18'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:37.048256 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-28ab2902-6cf9-4774-a3f5-39ffeeffe03a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e1f8b7984434ea1a0883feccec20d18'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:37.091874 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-28ab2902-6cf9-4774-a3f5-39ffeeffe03a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e1f8b7984434ea1a0883feccec20d18'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:37.103663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 816/1648] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:24:36 2022] DELETE /identity/v3/users/2e1f8b7984434ea1a0883feccec20d18 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:24:37.111281 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a7138-7554-4cd6-8c56-1e9b22ab9bff None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:37.112300 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a7138-7554-4cd6-8c56-1e9b22ab9bff None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:37.112836 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a7138-7554-4cd6-8c56-1e9b22ab9bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:37.146063 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-112a7138-7554-4cd6-8c56-1e9b22ab9bff None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:37.148109 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-112a7138-7554-4cd6-8c56-1e9b22ab9bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:37.408596 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab2902-6cf9-4774-a3f5-39ffeeffe03a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:37.414710 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-28ab2902-6cf9-4774-a3f5-39ffeeffe03a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:37.453278 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 833/1649] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:37.477471 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab2902-6cf9-4774-a3f5-39ffeeffe03a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:37.497168 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf2fce-202f-42c0-ba2e-f36afc99572c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:37.499075 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf2fce-202f-42c0-ba2e-f36afc99572c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:37.501042 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf2fce-202f-42c0-ba2e-f36afc99572c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:37.501636 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf2fce-202f-42c0-ba2e-f36afc99572c admin admin] PATH_INFO: `/v3/projects/32f923f138fb4564b7fd28431803fc3b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:37.506733 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf2fce-202f-42c0-ba2e-f36afc99572c admin admin] RBAC: Authorizing `identity:delete_project(project_id=32f923f138fb4564b7fd28431803fc3b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:37.515355 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf2fce-202f-42c0-ba2e-f36afc99572c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:37.528853 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a7138-7554-4cd6-8c56-1e9b22ab9bff None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:37.531269 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-112a7138-7554-4cd6-8c56-1e9b22ab9bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:37.570074 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:37.711164 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:37.727260 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a7138-7554-4cd6-8c56-1e9b22ab9bff None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:37.792093 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3adf2fce-202f-42c0-ba2e-f36afc99572c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32f923f138fb4564b7fd28431803fc3b', 'request_id': 'req-3adf2fce-202f-42c0-ba2e-f36afc99572c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:37.822592 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3adf2fce-202f-42c0-ba2e-f36afc99572c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32f923f138fb4564b7fd28431803fc3b', 'request_id': 'req-3adf2fce-202f-42c0-ba2e-f36afc99572c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:37.850969 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3adf2fce-202f-42c0-ba2e-f36afc99572c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32f923f138fb4564b7fd28431803fc3b', 'request_id': 'req-3adf2fce-202f-42c0-ba2e-f36afc99572c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:37.917137 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d2f20-4000-4ad7-af4a-3b5d29f8c547 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:37.917137 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d2f20-4000-4ad7-af4a-3b5d29f8c547 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:37.917137 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d2f20-4000-4ad7-af4a-3b5d29f8c547 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:37.918398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d2f20-4000-4ad7-af4a-3b5d29f8c547 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:37.944042 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3adf2fce-202f-42c0-ba2e-f36afc99572c admin admin] The token cache is being invalidate because project 32f923f138fb4564b7fd28431803fc3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:24:37.944737 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3adf2fce-202f-42c0-ba2e-f36afc99572c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:37.947248 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3adf2fce-202f-42c0-ba2e-f36afc99572c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:37.958409 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 817/1650] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:24:37 2022] DELETE /identity/v3/projects/32f923f138fb4564b7fd28431803fc3b => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:24:37.963824 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf2fce-202f-42c0-ba2e-f36afc99572c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:37.967318 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3adf2fce-202f-42c0-ba2e-f36afc99572c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:38.203322 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d2f20-4000-4ad7-af4a-3b5d29f8c547 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:38.217328 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d2f20-4000-4ad7-af4a-3b5d29f8c547 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:38.274794 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 834/1651] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:24:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 747 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:24:38.295018 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf2fce-202f-42c0-ba2e-f36afc99572c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:38.310528 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6169d5-b2f9-4e50-9639-9e92e999d3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:38.310833 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6169d5-b2f9-4e50-9639-9e92e999d3fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:38.311449 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6169d5-b2f9-4e50-9639-9e92e999d3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:38.311877 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6169d5-b2f9-4e50-9639-9e92e999d3fe admin admin] PATH_INFO: `/v3/users/0bbd03f51dac47ddb5f61a522ba7fb8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:38.313478 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6169d5-b2f9-4e50-9639-9e92e999d3fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bbd03f51dac47ddb5f61a522ba7fb8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:38.316897 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6169d5-b2f9-4e50-9639-9e92e999d3fe admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:38.434994 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4b6169d5-b2f9-4e50-9639-9e92e999d3fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bbd03f51dac47ddb5f61a522ba7fb8a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:38.452414 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4b6169d5-b2f9-4e50-9639-9e92e999d3fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bbd03f51dac47ddb5f61a522ba7fb8a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:38.483743 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4b6169d5-b2f9-4e50-9639-9e92e999d3fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bbd03f51dac47ddb5f61a522ba7fb8a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:38.487333 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4b6169d5-b2f9-4e50-9639-9e92e999d3fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bbd03f51dac47ddb5f61a522ba7fb8a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:38.552700 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4b6169d5-b2f9-4e50-9639-9e92e999d3fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bbd03f51dac47ddb5f61a522ba7fb8a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:38.562347 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 818/1652] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:24:37 2022] DELETE /identity/v3/users/0bbd03f51dac47ddb5f61a522ba7fb8a => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:24:41.251948 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d2f20-4000-4ad7-af4a-3b5d29f8c547 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:41.254091 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-756d2f20-4000-4ad7-af4a-3b5d29f8c547 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:41.303203 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d2f20-4000-4ad7-af4a-3b5d29f8c547 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:41.644915 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12c7164-5c2b-4739-a4b1-cd248c80abdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:41.648614 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c7164-5c2b-4739-a4b1-cd248c80abdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:41.649206 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c7164-5c2b-4739-a4b1-cd248c80abdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:41.649646 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c7164-5c2b-4739-a4b1-cd248c80abdb admin admin] PATH_INFO: `/v3/projects/2c772444cd3a43aba3efd52a54979d39` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:41.655222 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12c7164-5c2b-4739-a4b1-cd248c80abdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c772444cd3a43aba3efd52a54979d39)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:41.661914 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-c12c7164-5c2b-4739-a4b1-cd248c80abdb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 19:24:41.661914 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: warnings.warn(msg) Jul 25 19:24:41.661914 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: Jul 25 19:24:41.665596 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12c7164-5c2b-4739-a4b1-cd248c80abdb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:41.768684 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:41.931157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c12c7164-5c2b-4739-a4b1-cd248c80abdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c772444cd3a43aba3efd52a54979d39', 'request_id': 'req-c12c7164-5c2b-4739-a4b1-cd248c80abdb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:41.981609 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c12c7164-5c2b-4739-a4b1-cd248c80abdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c772444cd3a43aba3efd52a54979d39', 'request_id': 'req-c12c7164-5c2b-4739-a4b1-cd248c80abdb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:42.015781 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c12c7164-5c2b-4739-a4b1-cd248c80abdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c772444cd3a43aba3efd52a54979d39', 'request_id': 'req-c12c7164-5c2b-4739-a4b1-cd248c80abdb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:42.056322 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c12c7164-5c2b-4739-a4b1-cd248c80abdb admin admin] The token cache is being invalidate because project 2c772444cd3a43aba3efd52a54979d39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:24:42.056637 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c12c7164-5c2b-4739-a4b1-cd248c80abdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:42.057371 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c12c7164-5c2b-4739-a4b1-cd248c80abdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:42.061326 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 835/1653] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:24:41 2022] DELETE /identity/v3/projects/2c772444cd3a43aba3efd52a54979d39 => generated 0 bytes in 810 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:24:42.068929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed979712-72a1-4972-9075-5cc1b5d87e29 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:42.069945 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed979712-72a1-4972-9075-5cc1b5d87e29 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:42.070408 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed979712-72a1-4972-9075-5cc1b5d87e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:42.118663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ed979712-72a1-4972-9075-5cc1b5d87e29 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:42.121020 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ed979712-72a1-4972-9075-5cc1b5d87e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:42.205967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:42.349905 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59230a43-6126-42a6-bad9-38159df7a09a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:42.350693 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 819/1654] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:42.351767 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59230a43-6126-42a6-bad9-38159df7a09a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:42.352409 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59230a43-6126-42a6-bad9-38159df7a09a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:42.396047 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-59230a43-6126-42a6-bad9-38159df7a09a None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:42.401560 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-59230a43-6126-42a6-bad9-38159df7a09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:42.416658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 836/1655] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:24:42.423676 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed979712-72a1-4972-9075-5cc1b5d87e29 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:42.444245 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed979712-72a1-4972-9075-5cc1b5d87e29 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:42.459902 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c82a81-60af-4457-baf6-e53368dee58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:42.460609 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c82a81-60af-4457-baf6-e53368dee58f admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:42.460910 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c82a81-60af-4457-baf6-e53368dee58f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:42.461190 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c82a81-60af-4457-baf6-e53368dee58f admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:42.461972 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c82a81-60af-4457-baf6-e53368dee58f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:24:42.463621 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c82a81-60af-4457-baf6-e53368dee58f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:42.466670 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c82a81-60af-4457-baf6-e53368dee58f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:42.528403 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 820/1656] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:24:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:24:42.536170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59230a43-6126-42a6-bad9-38159df7a09a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:42.584763 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59230a43-6126-42a6-bad9-38159df7a09a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:42.615381 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fded4e-05d2-4314-8bab-6e0034195eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:42.616733 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fded4e-05d2-4314-8bab-6e0034195eb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:42.617235 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fded4e-05d2-4314-8bab-6e0034195eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:42.617766 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fded4e-05d2-4314-8bab-6e0034195eb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:42.619389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fded4e-05d2-4314-8bab-6e0034195eb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:42.625783 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fded4e-05d2-4314-8bab-6e0034195eb5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:42.731946 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c82a81-60af-4457-baf6-e53368dee58f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:42.736157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 837/1657] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:24:42 2022] POST /identity/v3/projects => generated 375 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:24:42.748100 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c82a81-60af-4457-baf6-e53368dee58f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:43.125721 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88610c8b-127c-4e94-a9a4-c93e2ed6837f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:43.131949 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88610c8b-127c-4e94-a9a4-c93e2ed6837f admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:43.140387 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88610c8b-127c-4e94-a9a4-c93e2ed6837f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:43.141134 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88610c8b-127c-4e94-a9a4-c93e2ed6837f admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:43.143234 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88610c8b-127c-4e94-a9a4-c93e2ed6837f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:43.156564 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88610c8b-127c-4e94-a9a4-c93e2ed6837f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:43.274404 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 821/1658] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:24:42 2022] POST /identity/v3/users => generated 288 bytes in 543 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:24:43.283046 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fded4e-05d2-4314-8bab-6e0034195eb5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:43.345357 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fded4e-05d2-4314-8bab-6e0034195eb5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:43.353270 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88610c8b-127c-4e94-a9a4-c93e2ed6837f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:43.364201 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-88610c8b-127c-4e94-a9a4-c93e2ed6837f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:43.376917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc6274-6fc2-4a7a-86da-aacc3a837dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:43.379222 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc6274-6fc2-4a7a-86da-aacc3a837dd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:43.379958 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc6274-6fc2-4a7a-86da-aacc3a837dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:43.380629 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc6274-6fc2-4a7a-86da-aacc3a837dd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:43.390413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfc6274-6fc2-4a7a-86da-aacc3a837dd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:43.395748 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfc6274-6fc2-4a7a-86da-aacc3a837dd8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:43.441797 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88610c8b-127c-4e94-a9a4-c93e2ed6837f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:43.481542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 838/1659] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:24:43 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:24:43.491476 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc6274-6fc2-4a7a-86da-aacc3a837dd8 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:43.502922 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc7ea68-7012-414d-b634-5041372d0000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:43.502922 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7ea68-7012-414d-b634-5041372d0000 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:43.518939 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7ea68-7012-414d-b634-5041372d0000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:43.527987 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7ea68-7012-414d-b634-5041372d0000 admin admin] PATH_INFO: `/v3/projects/833d164d4c6d4f6697dfb6dcbf88b4a6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:43.531691 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc7ea68-7012-414d-b634-5041372d0000 admin admin] RBAC: Authorizing `identity:delete_project(project_id=833d164d4c6d4f6697dfb6dcbf88b4a6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:43.540416 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc7ea68-7012-414d-b634-5041372d0000 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:43.565523 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc6274-6fc2-4a7a-86da-aacc3a837dd8 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:43.600510 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f176bace-58da-4f8e-b58b-12ebf9f59c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:43.602118 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f176bace-58da-4f8e-b58b-12ebf9f59c34 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:43.602846 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f176bace-58da-4f8e-b58b-12ebf9f59c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:43.603776 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f176bace-58da-4f8e-b58b-12ebf9f59c34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:43.606021 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f176bace-58da-4f8e-b58b-12ebf9f59c34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:43.610206 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f176bace-58da-4f8e-b58b-12ebf9f59c34 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:43.690219 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 839/1660] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:24:43 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:24:43.695801 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f176bace-58da-4f8e-b58b-12ebf9f59c34 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:43.739600 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f176bace-58da-4f8e-b58b-12ebf9f59c34 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:43.776036 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0cc7ea68-7012-414d-b634-5041372d0000 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '833d164d4c6d4f6697dfb6dcbf88b4a6', 'request_id': 'req-0cc7ea68-7012-414d-b634-5041372d0000'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:43.796671 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0cc7ea68-7012-414d-b634-5041372d0000 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '833d164d4c6d4f6697dfb6dcbf88b4a6', 'request_id': 'req-0cc7ea68-7012-414d-b634-5041372d0000'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:43.833408 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0cc7ea68-7012-414d-b634-5041372d0000 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '833d164d4c6d4f6697dfb6dcbf88b4a6', 'request_id': 'req-0cc7ea68-7012-414d-b634-5041372d0000'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:43.892206 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0cc7ea68-7012-414d-b634-5041372d0000 admin admin] The token cache is being invalidate because project 833d164d4c6d4f6697dfb6dcbf88b4a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:24:43.892813 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0cc7ea68-7012-414d-b634-5041372d0000 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:43.894958 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0cc7ea68-7012-414d-b634-5041372d0000 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:43.898845 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 822/1661] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:24:43 2022] DELETE /identity/v3/projects/833d164d4c6d4f6697dfb6dcbf88b4a6 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:24:43.911800 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2776f5d-26f6-4293-a013-961cdea6f2c2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:43.911800 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2776f5d-26f6-4293-a013-961cdea6f2c2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:43.911800 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2776f5d-26f6-4293-a013-961cdea6f2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:43.956070 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8078d36-e8ee-4faf-81df-e5c8f90c1a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:43.957733 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8078d36-e8ee-4faf-81df-e5c8f90c1a93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:43.958788 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8078d36-e8ee-4faf-81df-e5c8f90c1a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:43.960011 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8078d36-e8ee-4faf-81df-e5c8f90c1a93 admin admin] PATH_INFO: `/v3/projects/2e292f6f95e44b078843645b4c30ce4d/users/ac0fc5a408c74751b9db2ec078647b4d/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:44.050587 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e2776f5d-26f6-4293-a013-961cdea6f2c2 None None] MFA Rules not processed for user `ac2e584f88cd46f280b7fe04aa1382fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:44.051528 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8078d36-e8ee-4faf-81df-e5c8f90c1a93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e292f6f95e44b078843645b4c30ce4d, user_id=ac0fc5a408c74751b9db2ec078647b4d, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:44.053101 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e2776f5d-26f6-4293-a013-961cdea6f2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:44.063387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8078d36-e8ee-4faf-81df-e5c8f90c1a93 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:44.123677 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 840/1662] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:24:43 2022] PUT /identity/v3/projects/2e292f6f95e44b078843645b4c30ce4d/users/ac0fc5a408c74751b9db2ec078647b4d/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:24:44.130536 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5cc2d9-975b-46ba-acf2-02104f562b00 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:44.131841 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5cc2d9-975b-46ba-acf2-02104f562b00 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:44.133192 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5cc2d9-975b-46ba-acf2-02104f562b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:44.190273 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:44.232470 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-af5cc2d9-975b-46ba-acf2-02104f562b00 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:44.234567 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-af5cc2d9-975b-46ba-acf2-02104f562b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:44.424925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 823/1663] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 525 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:44.430632 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ee9a5-ab26-4df0-8821-054fb60d657f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:44.431066 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ee9a5-ab26-4df0-8821-054fb60d657f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:44.431461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ee9a5-ab26-4df0-8821-054fb60d657f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:44.457418 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:44.575871 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-933ee9a5-ab26-4df0-8821-054fb60d657f None None] MFA Rules not processed for user `ac0fc5a408c74751b9db2ec078647b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:44.578247 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-933ee9a5-ab26-4df0-8821-054fb60d657f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:44.623566 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 841/1664] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 499 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:44.628520 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5cc2d9-975b-46ba-acf2-02104f562b00 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:44.637496 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-af5cc2d9-975b-46ba-acf2-02104f562b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:44.856517 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 824/1665] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 429 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:44.863140 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69621c3e-a4d3-49a4-88b9-ffac3303d214 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:44.865130 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69621c3e-a4d3-49a4-88b9-ffac3303d214 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:44.865548 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69621c3e-a4d3-49a4-88b9-ffac3303d214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:44.915247 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-69621c3e-a4d3-49a4-88b9-ffac3303d214 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:44.915783 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-69621c3e-a4d3-49a4-88b9-ffac3303d214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:44.939755 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 825/1666] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:44.939755 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69621c3e-a4d3-49a4-88b9-ffac3303d214 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:44.946548 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-69621c3e-a4d3-49a4-88b9-ffac3303d214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:44.956694 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5cc2d9-975b-46ba-acf2-02104f562b00 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:44.981703 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69621c3e-a4d3-49a4-88b9-ffac3303d214 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:44.990240 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35953c1-c3ed-4092-b4d8-89b0b6f8726f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:44.992280 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35953c1-c3ed-4092-b4d8-89b0b6f8726f service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:44.992621 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35953c1-c3ed-4092-b4d8-89b0b6f8726f service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:44.992912 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35953c1-c3ed-4092-b4d8-89b0b6f8726f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:45.012862 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35953c1-c3ed-4092-b4d8-89b0b6f8726f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:45.013152 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b3015-bb98-432e-b326-c8bcdee963ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:45.014354 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35953c1-c3ed-4092-b4d8-89b0b6f8726f service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:45.025986 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b3015-bb98-432e-b326-c8bcdee963ac service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:45.025986 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b3015-bb98-432e-b326-c8bcdee963ac service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:45.025986 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b3015-bb98-432e-b326-c8bcdee963ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:45.041411 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 826/1667] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:24:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:24:45.044335 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35953c1-c3ed-4092-b4d8-89b0b6f8726f service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:45.070333 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35953c1-c3ed-4092-b4d8-89b0b6f8726f service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:45.093722 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453c6e2-56eb-4310-8f3b-1b5624397e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:45.094716 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453c6e2-56eb-4310-8f3b-1b5624397e73 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:45.095191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453c6e2-56eb-4310-8f3b-1b5624397e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:45.098620 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453c6e2-56eb-4310-8f3b-1b5624397e73 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:45.099734 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5453c6e2-56eb-4310-8f3b-1b5624397e73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:24:45.101947 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5453c6e2-56eb-4310-8f3b-1b5624397e73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:45.106619 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5453c6e2-56eb-4310-8f3b-1b5624397e73 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:45.172698 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 827/1668] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:24:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:24:45.174492 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453c6e2-56eb-4310-8f3b-1b5624397e73 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:45.228331 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453c6e2-56eb-4310-8f3b-1b5624397e73 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:45.241350 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e8b83f-7522-467f-948e-d2ee4084b942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:45.242415 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8b83f-7522-467f-948e-d2ee4084b942 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:45.243113 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8b83f-7522-467f-948e-d2ee4084b942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:45.243753 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8b83f-7522-467f-948e-d2ee4084b942 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:45.245119 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e8b83f-7522-467f-948e-d2ee4084b942 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:45.250598 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355b3015-bb98-432e-b326-c8bcdee963ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:45.253278 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355b3015-bb98-432e-b326-c8bcdee963ac service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:45.278852 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 842/1669] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:24:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 653 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:24:45.294470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e8b83f-7522-467f-948e-d2ee4084b942 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:45.379033 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b3015-bb98-432e-b326-c8bcdee963ac service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:45.379583 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 828/1670] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:24:45 2022] POST /identity/v3/projects => generated 373 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:24:45.411470 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b3015-bb98-432e-b326-c8bcdee963ac service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:45.603496 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444861c-97ea-406a-a064-d489bfb2c9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:45.606961 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444861c-97ea-406a-a064-d489bfb2c9de admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:45.606961 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444861c-97ea-406a-a064-d489bfb2c9de admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:45.606961 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444861c-97ea-406a-a064-d489bfb2c9de admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:45.607248 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444861c-97ea-406a-a064-d489bfb2c9de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:45.609453 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444861c-97ea-406a-a064-d489bfb2c9de admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:45.705431 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 843/1671] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:24:45 2022] POST /identity/v3/users => generated 287 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:24:45.710279 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e8b83f-7522-467f-948e-d2ee4084b942 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:45.731264 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e8b83f-7522-467f-948e-d2ee4084b942 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:45.753642 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40150b1e-4a7f-4c9a-b946-070473935921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:45.754710 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40150b1e-4a7f-4c9a-b946-070473935921 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:45.754868 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40150b1e-4a7f-4c9a-b946-070473935921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:45.755160 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40150b1e-4a7f-4c9a-b946-070473935921 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:45.755958 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40150b1e-4a7f-4c9a-b946-070473935921 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:45.769062 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40150b1e-4a7f-4c9a-b946-070473935921 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:45.787327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444861c-97ea-406a-a064-d489bfb2c9de admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:45.803604 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4444861c-97ea-406a-a064-d489bfb2c9de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:45.831831 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:45.841776 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 829/1672] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:24:45 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:24:45.843031 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40150b1e-4a7f-4c9a-b946-070473935921 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:45.843887 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444861c-97ea-406a-a064-d489bfb2c9de admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:45.862997 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c2eb9d-a4e5-4be7-82a5-bfd4281879d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:45.872350 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2eb9d-a4e5-4be7-82a5-bfd4281879d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:45.873158 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2eb9d-a4e5-4be7-82a5-bfd4281879d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:45.874855 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2eb9d-a4e5-4be7-82a5-bfd4281879d3 admin admin] PATH_INFO: `/v3/users/ac0fc5a408c74751b9db2ec078647b4d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:45.879268 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c2eb9d-a4e5-4be7-82a5-bfd4281879d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac0fc5a408c74751b9db2ec078647b4d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:45.887707 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c2eb9d-a4e5-4be7-82a5-bfd4281879d3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:45.889270 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40150b1e-4a7f-4c9a-b946-070473935921 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:45.909481 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08764f7-c452-42cc-80a5-5651b927d0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:45.910643 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08764f7-c452-42cc-80a5-5651b927d0e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:45.911151 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08764f7-c452-42cc-80a5-5651b927d0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:45.911692 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08764f7-c452-42cc-80a5-5651b927d0e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:45.912824 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08764f7-c452-42cc-80a5-5651b927d0e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:45.919142 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08764f7-c452-42cc-80a5-5651b927d0e8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:45.954160 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 830/1673] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:24:45 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:24:45.966279 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08764f7-c452-42cc-80a5-5651b927d0e8 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:46.009094 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08764f7-c452-42cc-80a5-5651b927d0e8 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:46.038249 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd5662-fd0e-43e1-b5d3-c5f27198e035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:46.042947 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd5662-fd0e-43e1-b5d3-c5f27198e035 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:46.043646 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd5662-fd0e-43e1-b5d3-c5f27198e035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:46.051571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd5662-fd0e-43e1-b5d3-c5f27198e035 admin admin] PATH_INFO: `/v3/projects/2dc7c299d00748aba3ebc7913277477b/users/c1feeebee42d4eefa64c057e53ae441a/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:46.125243 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bd5662-fd0e-43e1-b5d3-c5f27198e035 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dc7c299d00748aba3ebc7913277477b, user_id=c1feeebee42d4eefa64c057e53ae441a, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:46.129608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bd5662-fd0e-43e1-b5d3-c5f27198e035 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:46.140718 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-05c2eb9d-a4e5-4be7-82a5-bfd4281879d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac0fc5a408c74751b9db2ec078647b4d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:46.179784 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-05c2eb9d-a4e5-4be7-82a5-bfd4281879d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac0fc5a408c74751b9db2ec078647b4d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:46.186899 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 831/1674] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:24:45 2022] PUT /identity/v3/projects/2dc7c299d00748aba3ebc7913277477b/users/c1feeebee42d4eefa64c057e53ae441a/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:24:46.198135 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf66691-d0c7-40b4-97f9-c3ce1a45c28b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:46.198559 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf66691-d0c7-40b4-97f9-c3ce1a45c28b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:46.199299 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf66691-d0c7-40b4-97f9-c3ce1a45c28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:46.203396 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-05c2eb9d-a4e5-4be7-82a5-bfd4281879d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac0fc5a408c74751b9db2ec078647b4d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:46.204217 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-05c2eb9d-a4e5-4be7-82a5-bfd4281879d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac0fc5a408c74751b9db2ec078647b4d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:46.205092 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-05c2eb9d-a4e5-4be7-82a5-bfd4281879d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac0fc5a408c74751b9db2ec078647b4d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:46.279807 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 844/1675] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:24:45 2022] DELETE /identity/v3/users/ac0fc5a408c74751b9db2ec078647b4d => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:24:46.295164 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048483e5-f10c-4b64-bc04-1efca1e8e8b3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:46.295461 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048483e5-f10c-4b64-bc04-1efca1e8e8b3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:46.295814 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048483e5-f10c-4b64-bc04-1efca1e8e8b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:46.313465 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-acf66691-d0c7-40b4-97f9-c3ce1a45c28b None None] MFA Rules not processed for user `c1feeebee42d4eefa64c057e53ae441a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:46.314763 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-acf66691-d0c7-40b4-97f9-c3ce1a45c28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:46.341105 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:46.377691 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-048483e5-f10c-4b64-bc04-1efca1e8e8b3 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:46.379843 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-048483e5-f10c-4b64-bc04-1efca1e8e8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:46.619225 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 845/1676] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:24:46.662770 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 832/1677] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 466 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:24:46.663675 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048483e5-f10c-4b64-bc04-1efca1e8e8b3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:46.665844 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-048483e5-f10c-4b64-bc04-1efca1e8e8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:46.685954 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf66691-d0c7-40b4-97f9-c3ce1a45c28b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:46.689176 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-acf66691-d0c7-40b4-97f9-c3ce1a45c28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:46.691733 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048483e5-f10c-4b64-bc04-1efca1e8e8b3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:46.709161 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece61d79-0dff-4bf3-90e5-b9a3695f6b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:46.710061 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece61d79-0dff-4bf3-90e5-b9a3695f6b74 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:46.710439 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece61d79-0dff-4bf3-90e5-b9a3695f6b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:46.710853 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece61d79-0dff-4bf3-90e5-b9a3695f6b74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:46.711790 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece61d79-0dff-4bf3-90e5-b9a3695f6b74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:46.713986 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece61d79-0dff-4bf3-90e5-b9a3695f6b74 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:46.789721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 846/1678] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:24:46 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:24:46.799903 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece61d79-0dff-4bf3-90e5-b9a3695f6b74 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:47.011442 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf66691-d0c7-40b4-97f9-c3ce1a45c28b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:47.054408 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:47.075980 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece61d79-0dff-4bf3-90e5-b9a3695f6b74 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:47.096808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428b423-cb16-4d3f-b980-586ac10c33ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:47.097979 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428b423-cb16-4d3f-b980-586ac10c33ab admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:47.098571 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428b423-cb16-4d3f-b980-586ac10c33ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:47.099101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0428b423-cb16-4d3f-b980-586ac10c33ab admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:47.100262 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0428b423-cb16-4d3f-b980-586ac10c33ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:47.103912 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0428b423-cb16-4d3f-b980-586ac10c33ab admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:47.165442 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f621a-d71d-4830-8686-825a6061cee3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:47.166132 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f621a-d71d-4830-8686-825a6061cee3 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:47.166433 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f621a-d71d-4830-8686-825a6061cee3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:47.169589 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f621a-d71d-4830-8686-825a6061cee3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:47.191886 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 847/1679] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:24:46 2022] POST /identity/v3/users => generated 293 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:24:47.193926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428b423-cb16-4d3f-b980-586ac10c33ab admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:47.204199 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30f621a-d71d-4830-8686-825a6061cee3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:47.205740 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30f621a-d71d-4830-8686-825a6061cee3 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:47.218659 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0428b423-cb16-4d3f-b980-586ac10c33ab admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:47.231780 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591268fc-fb03-4f3e-9a9a-8b318fe7fcdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:47.232367 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591268fc-fb03-4f3e-9a9a-8b318fe7fcdf admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:47.232875 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591268fc-fb03-4f3e-9a9a-8b318fe7fcdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:47.233437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591268fc-fb03-4f3e-9a9a-8b318fe7fcdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:47.235628 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 833/1680] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:24:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:24:47.235817 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591268fc-fb03-4f3e-9a9a-8b318fe7fcdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:47.239177 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591268fc-fb03-4f3e-9a9a-8b318fe7fcdf admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:47.273406 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f621a-d71d-4830-8686-825a6061cee3 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:47.274674 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 848/1681] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:24:47.310842 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f621a-d71d-4830-8686-825a6061cee3 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:47.340497 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f504a6f-4aa6-4e10-8ce6-15a385b58c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:47.341623 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f504a6f-4aa6-4e10-8ce6-15a385b58c9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:47.343889 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591268fc-fb03-4f3e-9a9a-8b318fe7fcdf admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:47.349124 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f504a6f-4aa6-4e10-8ce6-15a385b58c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:47.349531 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f504a6f-4aa6-4e10-8ce6-15a385b58c9f admin admin] PATH_INFO: `/v3/projects/d8039387a2c6432393389b67aff32bfa/users/75bc08c45ea94c6f9f3f663cf705b15b/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:47.350686 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-591268fc-fb03-4f3e-9a9a-8b318fe7fcdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:47.405925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f504a6f-4aa6-4e10-8ce6-15a385b58c9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8039387a2c6432393389b67aff32bfa, user_id=75bc08c45ea94c6f9f3f663cf705b15b, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:47.409225 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f504a6f-4aa6-4e10-8ce6-15a385b58c9f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:47.417766 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591268fc-fb03-4f3e-9a9a-8b318fe7fcdf admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:47.442633 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f37ae8-f245-4217-8d73-0b0b3786e778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:47.446079 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f37ae8-f245-4217-8d73-0b0b3786e778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:47.447009 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f37ae8-f245-4217-8d73-0b0b3786e778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:47.449761 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 834/1682] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:24:47 2022] PUT /identity/v3/projects/d8039387a2c6432393389b67aff32bfa/users/75bc08c45ea94c6f9f3f663cf705b15b/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:24:47.456024 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f37ae8-f245-4217-8d73-0b0b3786e778 admin admin] PATH_INFO: `/v3/users/5f5358220a97443083b84a05ddbbad3f` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:47.456750 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f504a6f-4aa6-4e10-8ce6-15a385b58c9f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:47.461044 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f37ae8-f245-4217-8d73-0b0b3786e778 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f5358220a97443083b84a05ddbbad3f)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:47.473452 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f37ae8-f245-4217-8d73-0b0b3786e778 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:47.591347 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:47.639035 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f504a6f-4aa6-4e10-8ce6-15a385b58c9f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:47.655805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b949f-510c-4161-b36d-5a947f483483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:47.656577 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b949f-510c-4161-b36d-5a947f483483 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:47.656866 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b949f-510c-4161-b36d-5a947f483483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:47.657169 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b949f-510c-4161-b36d-5a947f483483 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:47.657926 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b949f-510c-4161-b36d-5a947f483483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:47.659920 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b949f-510c-4161-b36d-5a947f483483 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:47.695652 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 835/1683] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:24:47.703505 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b949f-510c-4161-b36d-5a947f483483 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:47.749413 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b949f-510c-4161-b36d-5a947f483483 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:47.770756 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca6a4b3-c78c-4665-9a5b-61fd0dd294de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:47.774804 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c2f37ae8-f245-4217-8d73-0b0b3786e778 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f5358220a97443083b84a05ddbbad3f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:47.780794 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6a4b3-c78c-4665-9a5b-61fd0dd294de admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:47.781283 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6a4b3-c78c-4665-9a5b-61fd0dd294de admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:47.781781 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6a4b3-c78c-4665-9a5b-61fd0dd294de admin admin] PATH_INFO: `/v3/projects/d8039387a2c6432393389b67aff32bfa/users/75bc08c45ea94c6f9f3f663cf705b15b/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:47.799023 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca6a4b3-c78c-4665-9a5b-61fd0dd294de admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8039387a2c6432393389b67aff32bfa, user_id=75bc08c45ea94c6f9f3f663cf705b15b, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:47.804699 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca6a4b3-c78c-4665-9a5b-61fd0dd294de admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:47.831066 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c2f37ae8-f245-4217-8d73-0b0b3786e778 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f5358220a97443083b84a05ddbbad3f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:47.846601 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 836/1684] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:24:47 2022] PUT /identity/v3/projects/d8039387a2c6432393389b67aff32bfa/users/75bc08c45ea94c6f9f3f663cf705b15b/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:24:47.872357 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb26be-b75c-4d38-9aa4-7c95a0383812 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:47.872642 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb26be-b75c-4d38-9aa4-7c95a0383812 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:47.873093 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb26be-b75c-4d38-9aa4-7c95a0383812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:47.877884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c2f37ae8-f245-4217-8d73-0b0b3786e778 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f5358220a97443083b84a05ddbbad3f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:47.878921 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c2f37ae8-f245-4217-8d73-0b0b3786e778 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f5358220a97443083b84a05ddbbad3f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:47.879826 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c2f37ae8-f245-4217-8d73-0b0b3786e778 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f5358220a97443083b84a05ddbbad3f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:47.963341 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 849/1685] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:24:47 2022] DELETE /identity/v3/users/5f5358220a97443083b84a05ddbbad3f => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:24:47.970161 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca296c-61c0-428c-b208-87856ff93e3a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:47.970742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca296c-61c0-428c-b208-87856ff93e3a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:47.971376 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca296c-61c0-428c-b208-87856ff93e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:47.990567 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5dcb26be-b75c-4d38-9aa4-7c95a0383812 None None] MFA Rules not processed for user `75bc08c45ea94c6f9f3f663cf705b15b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:47.992877 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5dcb26be-b75c-4d38-9aa4-7c95a0383812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:48.040418 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ffca296c-61c0-428c-b208-87856ff93e3a None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:48.046704 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ffca296c-61c0-428c-b208-87856ff93e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:48.306598 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 850/1686] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:48.337392 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 837/1687] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:47 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 468 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:24:48.361623 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca296c-61c0-428c-b208-87856ff93e3a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:48.363473 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ffca296c-61c0-428c-b208-87856ff93e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:48.402272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb26be-b75c-4d38-9aa4-7c95a0383812 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:48.404789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5dcb26be-b75c-4d38-9aa4-7c95a0383812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:48.540064 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:48.556242 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca296c-61c0-428c-b208-87856ff93e3a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:48.568738 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba5e4ba-8142-4d24-bb9f-e691a0d8f3c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:48.570325 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb26be-b75c-4d38-9aa4-7c95a0383812 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:48.571211 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5e4ba-8142-4d24-bb9f-e691a0d8f3c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:48.571648 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5e4ba-8142-4d24-bb9f-e691a0d8f3c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:48.572180 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5e4ba-8142-4d24-bb9f-e691a0d8f3c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:48.594651 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae795ac-2f14-471e-a96c-f7cb08469420 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:48.595841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae795ac-2f14-471e-a96c-f7cb08469420 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:48.597249 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae795ac-2f14-471e-a96c-f7cb08469420 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:48.597539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae795ac-2f14-471e-a96c-f7cb08469420 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:48.611097 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba5e4ba-8142-4d24-bb9f-e691a0d8f3c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:48.621959 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba5e4ba-8142-4d24-bb9f-e691a0d8f3c6 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:48.645983 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae795ac-2f14-471e-a96c-f7cb08469420 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:48.655005 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae795ac-2f14-471e-a96c-f7cb08469420 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:48.673249 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 838/1688] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:24:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:24:48.702672 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 851/1689] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:24:48 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:24:48.972356 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae795ac-2f14-471e-a96c-f7cb08469420 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:48.974216 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2ae795ac-2f14-471e-a96c-f7cb08469420 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:49.001231 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae795ac-2f14-471e-a96c-f7cb08469420 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:49.016909 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cdcdb2-3a8e-45d5-a714-a63d16e57391 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:49.018017 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cdcdb2-3a8e-45d5-a714-a63d16e57391 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:49.018313 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cdcdb2-3a8e-45d5-a714-a63d16e57391 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:49.018600 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cdcdb2-3a8e-45d5-a714-a63d16e57391 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:49.023386 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-58cdcdb2-3a8e-45d5-a714-a63d16e57391 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:49.143350 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:49.163182 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cdcdb2-3a8e-45d5-a714-a63d16e57391 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:49.164986 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cdcdb2-3a8e-45d5-a714-a63d16e57391 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:49.179363 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 839/1690] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:24:48 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:24:49.458106 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba5e4ba-8142-4d24-bb9f-e691a0d8f3c6 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:49.460887 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bba5e4ba-8142-4d24-bb9f-e691a0d8f3c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:49.511726 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cdcdb2-3a8e-45d5-a714-a63d16e57391 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:49.520412 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-58cdcdb2-3a8e-45d5-a714-a63d16e57391 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:49.563952 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba5e4ba-8142-4d24-bb9f-e691a0d8f3c6 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:49.595152 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e458a762-6060-46cc-b48b-688e72f30865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:49.598509 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458a762-6060-46cc-b48b-688e72f30865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:49.601037 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458a762-6060-46cc-b48b-688e72f30865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:49.602990 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458a762-6060-46cc-b48b-688e72f30865 admin admin] PATH_INFO: `/v3/projects/2e292f6f95e44b078843645b4c30ce4d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:49.607653 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e458a762-6060-46cc-b48b-688e72f30865 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e292f6f95e44b078843645b4c30ce4d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:49.608943 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cdcdb2-3a8e-45d5-a714-a63d16e57391 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:49.613119 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e458a762-6060-46cc-b48b-688e72f30865 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:49.628370 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c8c8ed-e777-4976-b1fd-4f7ec712339c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:49.633300 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8c8ed-e777-4976-b1fd-4f7ec712339c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:49.633960 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8c8ed-e777-4976-b1fd-4f7ec712339c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:49.634361 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8c8ed-e777-4976-b1fd-4f7ec712339c admin admin] PATH_INFO: `/v3/users/6f974d50b4ec41ddbf00e548606f2ba1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:49.637146 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c8c8ed-e777-4976-b1fd-4f7ec712339c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f974d50b4ec41ddbf00e548606f2ba1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:49.644036 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c8c8ed-e777-4976-b1fd-4f7ec712339c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:49.733118 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e458a762-6060-46cc-b48b-688e72f30865 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e292f6f95e44b078843645b4c30ce4d', 'request_id': 'req-e458a762-6060-46cc-b48b-688e72f30865'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:49.764038 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e458a762-6060-46cc-b48b-688e72f30865 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e292f6f95e44b078843645b4c30ce4d', 'request_id': 'req-e458a762-6060-46cc-b48b-688e72f30865'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:49.765213 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-52c8c8ed-e777-4976-b1fd-4f7ec712339c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f974d50b4ec41ddbf00e548606f2ba1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:49.783071 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-52c8c8ed-e777-4976-b1fd-4f7ec712339c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f974d50b4ec41ddbf00e548606f2ba1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:49.784762 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e458a762-6060-46cc-b48b-688e72f30865 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e292f6f95e44b078843645b4c30ce4d', 'request_id': 'req-e458a762-6060-46cc-b48b-688e72f30865'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:49.811008 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-52c8c8ed-e777-4976-b1fd-4f7ec712339c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f974d50b4ec41ddbf00e548606f2ba1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:49.818488 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-52c8c8ed-e777-4976-b1fd-4f7ec712339c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f974d50b4ec41ddbf00e548606f2ba1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:49.834443 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e458a762-6060-46cc-b48b-688e72f30865 admin admin] The token cache is being invalidate because project 2e292f6f95e44b078843645b4c30ce4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:24:49.836657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e458a762-6060-46cc-b48b-688e72f30865 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:49.837968 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e458a762-6060-46cc-b48b-688e72f30865 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:49.845429 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 852/1691] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:24:49 2022] DELETE /identity/v3/projects/2e292f6f95e44b078843645b4c30ce4d => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:24:49.855379 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aadb40-ab79-4ee7-81cd-bf7b87658863 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:49.856101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aadb40-ab79-4ee7-81cd-bf7b87658863 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:49.856613 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aadb40-ab79-4ee7-81cd-bf7b87658863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:49.859323 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-52c8c8ed-e777-4976-b1fd-4f7ec712339c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f974d50b4ec41ddbf00e548606f2ba1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:49.873699 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 840/1692] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:24:49 2022] DELETE /identity/v3/users/6f974d50b4ec41ddbf00e548606f2ba1 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:24:49.878499 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bf19f-b746-4d9b-b4a3-90d41eac3e6f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:49.879027 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bf19f-b746-4d9b-b4a3-90d41eac3e6f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:49.879479 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bf19f-b746-4d9b-b4a3-90d41eac3e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:49.910546 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5b6bf19f-b746-4d9b-b4a3-90d41eac3e6f None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:49.912149 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5b6bf19f-b746-4d9b-b4a3-90d41eac3e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:49.938942 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-97aadb40-ab79-4ee7-81cd-bf7b87658863 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:49.942059 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-97aadb40-ab79-4ee7-81cd-bf7b87658863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:50.094666 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 841/1693] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:50.111590 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6bf19f-b746-4d9b-b4a3-90d41eac3e6f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:50.117140 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5b6bf19f-b746-4d9b-b4a3-90d41eac3e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:50.150117 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 853/1694] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:50.179406 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa580a-f01e-491d-8167-51afed840f2d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:50.180272 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa580a-f01e-491d-8167-51afed840f2d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:50.181572 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa580a-f01e-491d-8167-51afed840f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:50.230940 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d8fa580a-f01e-491d-8167-51afed840f2d None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:50.237764 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d8fa580a-f01e-491d-8167-51afed840f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:50.279925 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 854/1695] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:24:50.286940 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa580a-f01e-491d-8167-51afed840f2d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:50.346118 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa580a-f01e-491d-8167-51afed840f2d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:50.354955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf01f087-4570-42d8-87b1-2f7184d8e9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:50.355842 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01f087-4570-42d8-87b1-2f7184d8e9cd admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:50.356193 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01f087-4570-42d8-87b1-2f7184d8e9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:50.356617 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01f087-4570-42d8-87b1-2f7184d8e9cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:50.357798 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf01f087-4570-42d8-87b1-2f7184d8e9cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:24:50.358343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf01f087-4570-42d8-87b1-2f7184d8e9cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:50.365521 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf01f087-4570-42d8-87b1-2f7184d8e9cd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:50.380567 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6bf19f-b746-4d9b-b4a3-90d41eac3e6f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:50.389294 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 855/1696] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:24:50.391448 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970a559-a80e-49ea-8ba4-5c3505f29696 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:50.393040 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970a559-a80e-49ea-8ba4-5c3505f29696 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:50.393752 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970a559-a80e-49ea-8ba4-5c3505f29696 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:50.394302 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970a559-a80e-49ea-8ba4-5c3505f29696 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:50.396654 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf01f087-4570-42d8-87b1-2f7184d8e9cd admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:50.422019 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970a559-a80e-49ea-8ba4-5c3505f29696 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:50.424470 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf01f087-4570-42d8-87b1-2f7184d8e9cd admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:50.426193 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970a559-a80e-49ea-8ba4-5c3505f29696 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:50.449447 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30475d-b1fd-4cbd-a073-bf0055974770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:50.449447 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30475d-b1fd-4cbd-a073-bf0055974770 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:50.451026 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30475d-b1fd-4cbd-a073-bf0055974770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:50.451026 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30475d-b1fd-4cbd-a073-bf0055974770 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:50.456244 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30475d-b1fd-4cbd-a073-bf0055974770 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:50.456722 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 842/1697] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:24:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:24:50.458279 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30475d-b1fd-4cbd-a073-bf0055974770 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:50.511341 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 856/1698] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:24:50 2022] POST /identity/v3/projects => generated 363 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:24:50.512514 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970a559-a80e-49ea-8ba4-5c3505f29696 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:50.523873 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:50.531976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970a559-a80e-49ea-8ba4-5c3505f29696 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:50.685291 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752b1b37-ff67-4689-a762-f25e21308e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:50.686383 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b1b37-ff67-4689-a762-f25e21308e4a admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:50.686778 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b1b37-ff67-4689-a762-f25e21308e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:50.687165 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b1b37-ff67-4689-a762-f25e21308e4a admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:50.688122 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752b1b37-ff67-4689-a762-f25e21308e4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:50.691030 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752b1b37-ff67-4689-a762-f25e21308e4a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:50.732730 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 843/1699] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:24:50 2022] POST /identity/v3/users => generated 282 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:24:50.738656 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30475d-b1fd-4cbd-a073-bf0055974770 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:50.762060 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30475d-b1fd-4cbd-a073-bf0055974770 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:50.784787 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec16eac-8109-41ee-aefe-311731714d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:50.785523 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec16eac-8109-41ee-aefe-311731714d0c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:50.785812 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec16eac-8109-41ee-aefe-311731714d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:50.786094 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec16eac-8109-41ee-aefe-311731714d0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:50.786874 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec16eac-8109-41ee-aefe-311731714d0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:50.788662 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec16eac-8109-41ee-aefe-311731714d0c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:50.812170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 857/1700] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:24:50.818464 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752b1b37-ff67-4689-a762-f25e21308e4a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:50.848141 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752b1b37-ff67-4689-a762-f25e21308e4a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:50.875655 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e62d0-6c13-4bd3-af36-e9ed433a3dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:50.880576 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e62d0-6c13-4bd3-af36-e9ed433a3dac admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:50.881304 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e62d0-6c13-4bd3-af36-e9ed433a3dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:50.881821 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e62d0-6c13-4bd3-af36-e9ed433a3dac admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:50.884921 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e62d0-6c13-4bd3-af36-e9ed433a3dac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:50.892166 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e62d0-6c13-4bd3-af36-e9ed433a3dac admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:50.928025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 844/1701] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:24:50.934095 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec16eac-8109-41ee-aefe-311731714d0c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:50.950337 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec16eac-8109-41ee-aefe-311731714d0c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:50.972645 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e30664-1c88-43cb-8b1e-29b92b0133ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:50.974206 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e30664-1c88-43cb-8b1e-29b92b0133ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:50.974206 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e30664-1c88-43cb-8b1e-29b92b0133ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:50.974206 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e30664-1c88-43cb-8b1e-29b92b0133ab admin admin] PATH_INFO: `/v3/projects/498b69be7a894313aaf5016790b30d4c/users/720004e5661c4843aa6f016ce41ff76a/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:51.010340 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e30664-1c88-43cb-8b1e-29b92b0133ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=498b69be7a894313aaf5016790b30d4c, user_id=720004e5661c4843aa6f016ce41ff76a, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:51.013864 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e30664-1c88-43cb-8b1e-29b92b0133ab admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:51.027310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:51.049524 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 858/1702] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:24:50 2022] PUT /identity/v3/projects/498b69be7a894313aaf5016790b30d4c/users/720004e5661c4843aa6f016ce41ff76a/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:24:51.069279 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719de2f-9db7-48d9-b920-ad4284e54549 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:51.070395 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719de2f-9db7-48d9-b920-ad4284e54549 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:51.070966 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0719de2f-9db7-48d9-b920-ad4284e54549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:51.210605 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0719de2f-9db7-48d9-b920-ad4284e54549 None None] MFA Rules not processed for user `720004e5661c4843aa6f016ce41ff76a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:51.218578 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0719de2f-9db7-48d9-b920-ad4284e54549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:51.648915 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 845/1703] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:51 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 580 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:51.649964 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e30664-1c88-43cb-8b1e-29b92b0133ab admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:51.941634 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e30664-1c88-43cb-8b1e-29b92b0133ab admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:51.958086 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be640aa-8144-4819-a6ce-f1264419063c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:51.959310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be640aa-8144-4819-a6ce-f1264419063c admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:51.962138 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be640aa-8144-4819-a6ce-f1264419063c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:51.962660 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be640aa-8144-4819-a6ce-f1264419063c admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:51.963573 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be640aa-8144-4819-a6ce-f1264419063c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:51.972431 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be640aa-8144-4819-a6ce-f1264419063c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:52.045518 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 859/1704] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:24:51 2022] POST /identity/v3/projects => generated 363 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:24:52.052108 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0719de2f-9db7-48d9-b920-ad4284e54549 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:52.108009 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0719de2f-9db7-48d9-b920-ad4284e54549 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:52.416820 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f00054-e082-4af8-a3c5-b1e9a1512d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:52.418332 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f00054-e082-4af8-a3c5-b1e9a1512d9b admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:52.419001 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f00054-e082-4af8-a3c5-b1e9a1512d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:52.419698 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f00054-e082-4af8-a3c5-b1e9a1512d9b admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:52.421276 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f00054-e082-4af8-a3c5-b1e9a1512d9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:52.424436 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f00054-e082-4af8-a3c5-b1e9a1512d9b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:52.478921 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 846/1705] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:24:52 2022] POST /identity/v3/users => generated 288 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:24:52.486181 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be640aa-8144-4819-a6ce-f1264419063c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:52.518894 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be640aa-8144-4819-a6ce-f1264419063c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:52.537780 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f692c6e-5680-45fe-914e-d10d267242b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:52.538660 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f692c6e-5680-45fe-914e-d10d267242b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:52.538970 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f692c6e-5680-45fe-914e-d10d267242b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:52.539357 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f692c6e-5680-45fe-914e-d10d267242b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:52.540171 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f692c6e-5680-45fe-914e-d10d267242b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:52.542457 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f692c6e-5680-45fe-914e-d10d267242b3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:52.575237 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 860/1706] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:24:52.582269 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f00054-e082-4af8-a3c5-b1e9a1512d9b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:52.588198 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:52.600699 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f00054-e082-4af8-a3c5-b1e9a1512d9b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:52.614680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c353c1-9813-4df5-b768-561eadebbb30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:52.615483 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c353c1-9813-4df5-b768-561eadebbb30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:52.615811 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c353c1-9813-4df5-b768-561eadebbb30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:52.616189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c353c1-9813-4df5-b768-561eadebbb30 admin admin] PATH_INFO: `/v3/projects/19f9bf95e41f439c8956f250859575f0/users/d2eb8262d2164fdf8303f443439689ac/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:52.654794 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c353c1-9813-4df5-b768-561eadebbb30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19f9bf95e41f439c8956f250859575f0, user_id=d2eb8262d2164fdf8303f443439689ac, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:52.657981 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c353c1-9813-4df5-b768-561eadebbb30 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:52.688025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 847/1707] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:24:52 2022] PUT /identity/v3/projects/19f9bf95e41f439c8956f250859575f0/users/d2eb8262d2164fdf8303f443439689ac/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:24:52.693920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f692c6e-5680-45fe-914e-d10d267242b3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:52.729063 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f692c6e-5680-45fe-914e-d10d267242b3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:52.838426 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b52224-7040-414e-9da0-43272809330c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:52.839611 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b52224-7040-414e-9da0-43272809330c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:52.839978 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b52224-7040-414e-9da0-43272809330c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:52.841176 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b52224-7040-414e-9da0-43272809330c admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:52.844018 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b52224-7040-414e-9da0-43272809330c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:52.848212 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b52224-7040-414e-9da0-43272809330c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:52.869347 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 861/1708] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:24:52.876392 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c353c1-9813-4df5-b768-561eadebbb30 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:52.893985 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c353c1-9813-4df5-b768-561eadebbb30 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:52.913592 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1081d11b-cdbb-46b7-8fa0-b99c3e7d3e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:52.915409 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081d11b-cdbb-46b7-8fa0-b99c3e7d3e85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:52.916309 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081d11b-cdbb-46b7-8fa0-b99c3e7d3e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:52.919606 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081d11b-cdbb-46b7-8fa0-b99c3e7d3e85 admin admin] PATH_INFO: `/v3/projects/19f9bf95e41f439c8956f250859575f0/users/d2eb8262d2164fdf8303f443439689ac/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:52.930165 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1081d11b-cdbb-46b7-8fa0-b99c3e7d3e85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19f9bf95e41f439c8956f250859575f0, user_id=d2eb8262d2164fdf8303f443439689ac, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:52.948738 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1081d11b-cdbb-46b7-8fa0-b99c3e7d3e85 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:52.993385 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 848/1709] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:24:52 2022] PUT /identity/v3/projects/19f9bf95e41f439c8956f250859575f0/users/d2eb8262d2164fdf8303f443439689ac/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:24:53.019349 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd938d0e-4e6a-4e74-8d84-c59597d6d680 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:53.019740 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd938d0e-4e6a-4e74-8d84-c59597d6d680 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:53.019987 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd938d0e-4e6a-4e74-8d84-c59597d6d680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:53.064731 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1081d11b-cdbb-46b7-8fa0-b99c3e7d3e85 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:53.067549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1081d11b-cdbb-46b7-8fa0-b99c3e7d3e85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:53.087452 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-cd938d0e-4e6a-4e74-8d84-c59597d6d680 None None] MFA Rules not processed for user `d2eb8262d2164fdf8303f443439689ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:53.090785 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cd938d0e-4e6a-4e74-8d84-c59597d6d680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:53.239364 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:53.299702 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1081d11b-cdbb-46b7-8fa0-b99c3e7d3e85 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:53.321171 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f45879-41f2-4cb1-bd29-f50ecb8ef6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:53.324279 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f45879-41f2-4cb1-bd29-f50ecb8ef6b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:53.324279 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f45879-41f2-4cb1-bd29-f50ecb8ef6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:53.327066 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f45879-41f2-4cb1-bd29-f50ecb8ef6b7 admin admin] PATH_INFO: `/v3/projects/4869e3056eb340c7a6a45949b97a8e7c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:53.335797 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f45879-41f2-4cb1-bd29-f50ecb8ef6b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4869e3056eb340c7a6a45949b97a8e7c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:53.347345 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 862/1710] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:53 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:53.349288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f45879-41f2-4cb1-bd29-f50ecb8ef6b7 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:53.356191 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd938d0e-4e6a-4e74-8d84-c59597d6d680 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:53.361816 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cd938d0e-4e6a-4e74-8d84-c59597d6d680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:53.572236 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c7f45879-41f2-4cb1-bd29-f50ecb8ef6b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4869e3056eb340c7a6a45949b97a8e7c', 'request_id': 'req-c7f45879-41f2-4cb1-bd29-f50ecb8ef6b7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:53.580413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd938d0e-4e6a-4e74-8d84-c59597d6d680 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:53.617476 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c7f45879-41f2-4cb1-bd29-f50ecb8ef6b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4869e3056eb340c7a6a45949b97a8e7c', 'request_id': 'req-c7f45879-41f2-4cb1-bd29-f50ecb8ef6b7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:53.649977 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c7f45879-41f2-4cb1-bd29-f50ecb8ef6b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4869e3056eb340c7a6a45949b97a8e7c', 'request_id': 'req-c7f45879-41f2-4cb1-bd29-f50ecb8ef6b7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:53.702501 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c7f45879-41f2-4cb1-bd29-f50ecb8ef6b7 admin admin] The token cache is being invalidate because project 4869e3056eb340c7a6a45949b97a8e7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:24:53.706050 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c7f45879-41f2-4cb1-bd29-f50ecb8ef6b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:53.712443 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c7f45879-41f2-4cb1-bd29-f50ecb8ef6b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:53.717441 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 849/1711] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:24:53 2022] DELETE /identity/v3/projects/4869e3056eb340c7a6a45949b97a8e7c => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:24:53.735730 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9e85a-38e8-4b24-80fa-51a16b36069f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:53.736211 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9e85a-38e8-4b24-80fa-51a16b36069f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:53.737083 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9e85a-38e8-4b24-80fa-51a16b36069f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:53.832927 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-62f9e85a-38e8-4b24-80fa-51a16b36069f None None] MFA Rules not processed for user `ac2e584f88cd46f280b7fe04aa1382fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:53.834475 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-62f9e85a-38e8-4b24-80fa-51a16b36069f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:53.875652 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf88486-8f2d-40d0-82f8-9825f17bc16f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:53.877613 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf88486-8f2d-40d0-82f8-9825f17bc16f service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:53.880224 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf88486-8f2d-40d0-82f8-9825f17bc16f service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:53.881087 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf88486-8f2d-40d0-82f8-9825f17bc16f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:53.895079 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4cf88486-8f2d-40d0-82f8-9825f17bc16f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:54.088276 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:54.119172 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf88486-8f2d-40d0-82f8-9825f17bc16f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:54.122072 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf88486-8f2d-40d0-82f8-9825f17bc16f service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:54.130445 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 850/1712] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 412 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:54.134359 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce66ac-780d-4e91-a879-560ee9dc39fa None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:54.134935 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce66ac-780d-4e91-a879-560ee9dc39fa None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:54.139837 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce66ac-780d-4e91-a879-560ee9dc39fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:54.171279 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 863/1713] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:24:53 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 817 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:24:54.172286 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf88486-8f2d-40d0-82f8-9825f17bc16f service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:54.185150 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4cf88486-8f2d-40d0-82f8-9825f17bc16f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:54.238075 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8bce66ac-780d-4e91-a879-560ee9dc39fa None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:54.239362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8bce66ac-780d-4e91-a879-560ee9dc39fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:54.415547 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 851/1714] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:54.420449 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49173566-2073-41c8-9f29-25bea492f9e5 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:54.420449 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49173566-2073-41c8-9f29-25bea492f9e5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:54.420449 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49173566-2073-41c8-9f29-25bea492f9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:54.439470 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:54.500375 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-49173566-2073-41c8-9f29-25bea492f9e5 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:54.502581 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-49173566-2073-41c8-9f29-25bea492f9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:54.503501 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf88486-8f2d-40d0-82f8-9825f17bc16f service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:54.541562 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67839d4-3e20-4758-a0c9-6450c9b7b565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:54.543679 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67839d4-3e20-4758-a0c9-6450c9b7b565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:54.550242 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67839d4-3e20-4758-a0c9-6450c9b7b565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:54.550700 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67839d4-3e20-4758-a0c9-6450c9b7b565 admin admin] PATH_INFO: `/v3/projects/70b28a4870d94263bf7a134130078181` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:54.553676 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 852/1715] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:54.565097 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49173566-2073-41c8-9f29-25bea492f9e5 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:54.571294 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67839d4-3e20-4758-a0c9-6450c9b7b565 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70b28a4870d94263bf7a134130078181)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:54.572436 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67839d4-3e20-4758-a0c9-6450c9b7b565 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:54.640562 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49173566-2073-41c8-9f29-25bea492f9e5 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:54.709889 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12c523-1f3d-40f1-87ed-e41d62679025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:54.717255 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12c523-1f3d-40f1-87ed-e41d62679025 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:54.727226 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12c523-1f3d-40f1-87ed-e41d62679025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:54.736341 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12c523-1f3d-40f1-87ed-e41d62679025 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:54.736341 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12c523-1f3d-40f1-87ed-e41d62679025 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:24:54.738100 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12c523-1f3d-40f1-87ed-e41d62679025 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:54.757273 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12c523-1f3d-40f1-87ed-e41d62679025 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:54.775424 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e67839d4-3e20-4758-a0c9-6450c9b7b565 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70b28a4870d94263bf7a134130078181', 'request_id': 'req-e67839d4-3e20-4758-a0c9-6450c9b7b565'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:54.822429 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 853/1716] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:24:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:24:54.826277 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e67839d4-3e20-4758-a0c9-6450c9b7b565 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70b28a4870d94263bf7a134130078181', 'request_id': 'req-e67839d4-3e20-4758-a0c9-6450c9b7b565'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:54.828400 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12c523-1f3d-40f1-87ed-e41d62679025 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:54.869428 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12c523-1f3d-40f1-87ed-e41d62679025 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:54.894921 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e67839d4-3e20-4758-a0c9-6450c9b7b565 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70b28a4870d94263bf7a134130078181', 'request_id': 'req-e67839d4-3e20-4758-a0c9-6450c9b7b565'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:54.974315 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e67839d4-3e20-4758-a0c9-6450c9b7b565 admin admin] The token cache is being invalidate because project 70b28a4870d94263bf7a134130078181 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:24:54.975861 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e67839d4-3e20-4758-a0c9-6450c9b7b565 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:54.992310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e67839d4-3e20-4758-a0c9-6450c9b7b565 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:55.015098 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 864/1717] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:24:54 2022] DELETE /identity/v3/projects/70b28a4870d94263bf7a134130078181 => generated 0 bytes in 833 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:24:55.019122 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67839d4-3e20-4758-a0c9-6450c9b7b565 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:55.024935 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e67839d4-3e20-4758-a0c9-6450c9b7b565 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:55.373815 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6689505-a6f4-4e87-882d-7a7318ce87d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:55.375037 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6689505-a6f4-4e87-882d-7a7318ce87d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:55.375488 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6689505-a6f4-4e87-882d-7a7318ce87d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:55.394476 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6689505-a6f4-4e87-882d-7a7318ce87d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:55.395717 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6689505-a6f4-4e87-882d-7a7318ce87d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:55.402205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6689505-a6f4-4e87-882d-7a7318ce87d7 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:55.427485 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67839d4-3e20-4758-a0c9-6450c9b7b565 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:55.451343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58db4a-f65b-4b04-8e23-77da3a852a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:55.455557 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58db4a-f65b-4b04-8e23-77da3a852a96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:55.457348 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58db4a-f65b-4b04-8e23-77da3a852a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:55.458070 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58db4a-f65b-4b04-8e23-77da3a852a96 admin admin] PATH_INFO: `/v3/users/447def5b8e6f40bab08e0459b03ec9b3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:55.462016 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b58db4a-f65b-4b04-8e23-77da3a852a96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=447def5b8e6f40bab08e0459b03ec9b3)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:55.475347 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b58db4a-f65b-4b04-8e23-77da3a852a96 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:55.535389 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 854/1718] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:24:54 2022] POST /identity/v3/projects => generated 365 bytes in 704 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:24:55.538578 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6689505-a6f4-4e87-882d-7a7318ce87d7 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:55.554511 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d6689505-a6f4-4e87-882d-7a7318ce87d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:55.771311 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1b58db4a-f65b-4b04-8e23-77da3a852a96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '447def5b8e6f40bab08e0459b03ec9b3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:55.814962 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1b58db4a-f65b-4b04-8e23-77da3a852a96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '447def5b8e6f40bab08e0459b03ec9b3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:55.854418 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1b58db4a-f65b-4b04-8e23-77da3a852a96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '447def5b8e6f40bab08e0459b03ec9b3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:55.867164 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1b58db4a-f65b-4b04-8e23-77da3a852a96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '447def5b8e6f40bab08e0459b03ec9b3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:55.868541 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1b58db4a-f65b-4b04-8e23-77da3a852a96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '447def5b8e6f40bab08e0459b03ec9b3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:55.954631 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6689505-a6f4-4e87-882d-7a7318ce87d7 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:55.972922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 865/1719] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:24:55 2022] DELETE /identity/v3/users/447def5b8e6f40bab08e0459b03ec9b3 => generated 0 bytes in 952 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:24:55.986239 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a98eaf-8bb3-424e-8aa6-b3957f999871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:55.987261 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a98eaf-8bb3-424e-8aa6-b3957f999871 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:55.988116 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a98eaf-8bb3-424e-8aa6-b3957f999871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:55.988398 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a98eaf-8bb3-424e-8aa6-b3957f999871 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:55.989562 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a98eaf-8bb3-424e-8aa6-b3957f999871 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:56.004483 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a98eaf-8bb3-424e-8aa6-b3957f999871 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:56.046489 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:56.124955 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 855/1720] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:24:55 2022] POST /identity/v3/users => generated 283 bytes in 584 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:24:56.127203 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58db4a-f65b-4b04-8e23-77da3a852a96 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:56.130008 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1b58db4a-f65b-4b04-8e23-77da3a852a96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:56.161560 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea37c96-5624-400e-8581-d9a70d4a1b13 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:56.162019 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea37c96-5624-400e-8581-d9a70d4a1b13 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:56.162185 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea37c96-5624-400e-8581-d9a70d4a1b13 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:56.175717 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 856/1721] 10.222.0.18 () {54 vars in 912 bytes} [Mon Jul 25 19:24:56 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:24:56.183061 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:56.187640 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3ce75-f13b-427c-8573-6aeaf1cb16d3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:56.187843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3ce75-f13b-427c-8573-6aeaf1cb16d3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:56.187960 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3ce75-f13b-427c-8573-6aeaf1cb16d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:56.409775 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-99c3ce75-f13b-427c-8573-6aeaf1cb16d3 None None] MFA Rules not processed for user `181c84715c094100b414cea3c4919eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:56.414404 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-99c3ce75-f13b-427c-8573-6aeaf1cb16d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:56.502552 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58db4a-f65b-4b04-8e23-77da3a852a96 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:56.541240 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a516b2e-22dc-4adb-a229-af51e26b0a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:56.543020 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a516b2e-22dc-4adb-a229-af51e26b0a1e admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:56.543587 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a516b2e-22dc-4adb-a229-af51e26b0a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:56.544112 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a516b2e-22dc-4adb-a229-af51e26b0a1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:56.545476 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a516b2e-22dc-4adb-a229-af51e26b0a1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:56.549965 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a516b2e-22dc-4adb-a229-af51e26b0a1e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:56.607942 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 866/1722] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:24:56 2022] GET /identity/v3/roles => generated 1420 bytes in 482 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:24:56.640812 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a516b2e-22dc-4adb-a229-af51e26b0a1e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:56.648752 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0a516b2e-22dc-4adb-a229-af51e26b0a1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:56.717972 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 857/1723] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Jul 25 19:24:56 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 532 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:24:56.719331 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3ce75-f13b-427c-8573-6aeaf1cb16d3 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:56.732309 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a516b2e-22dc-4adb-a229-af51e26b0a1e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:56.760484 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a04af6c-fb4a-48e5-b2d7-f3d54cab5aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:56.760484 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04af6c-fb4a-48e5-b2d7-f3d54cab5aa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:56.761159 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04af6c-fb4a-48e5-b2d7-f3d54cab5aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:56.763381 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04af6c-fb4a-48e5-b2d7-f3d54cab5aa6 admin admin] PATH_INFO: `/v3/users/720004e5661c4843aa6f016ce41ff76a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:56.765014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3ce75-f13b-427c-8573-6aeaf1cb16d3 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:56.767958 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a04af6c-fb4a-48e5-b2d7-f3d54cab5aa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=720004e5661c4843aa6f016ce41ff76a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:56.779278 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a04af6c-fb4a-48e5-b2d7-f3d54cab5aa6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:56.786820 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9a9db7-fcf0-4b96-97d3-79472ee9d52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:56.789298 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a9db7-fcf0-4b96-97d3-79472ee9d52c admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:56.789769 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a9db7-fcf0-4b96-97d3-79472ee9d52c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:56.790244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a9db7-fcf0-4b96-97d3-79472ee9d52c admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:56.792084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9a9db7-fcf0-4b96-97d3-79472ee9d52c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:56.801170 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9a9db7-fcf0-4b96-97d3-79472ee9d52c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:56.853283 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 858/1724] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:24:56 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:24:56.857508 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9a9db7-fcf0-4b96-97d3-79472ee9d52c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:56.865415 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8b9a9db7-fcf0-4b96-97d3-79472ee9d52c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:56.920026 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9a9db7-fcf0-4b96-97d3-79472ee9d52c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:56.935726 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b17b16-06f4-4c3f-9752-982b122cf495 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:56.937895 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b17b16-06f4-4c3f-9752-982b122cf495 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:56.938684 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b17b16-06f4-4c3f-9752-982b122cf495 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:56.939108 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b17b16-06f4-4c3f-9752-982b122cf495 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:56.961215 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b17b16-06f4-4c3f-9752-982b122cf495 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:56.964548 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b17b16-06f4-4c3f-9752-982b122cf495 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:56.970785 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8a04af6c-fb4a-48e5-b2d7-f3d54cab5aa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '720004e5661c4843aa6f016ce41ff76a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:56.996970 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8a04af6c-fb4a-48e5-b2d7-f3d54cab5aa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '720004e5661c4843aa6f016ce41ff76a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:57.000252 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 859/1725] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:24:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:24:57.001839 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b17b16-06f4-4c3f-9752-982b122cf495 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:57.020422 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8a04af6c-fb4a-48e5-b2d7-f3d54cab5aa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '720004e5661c4843aa6f016ce41ff76a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:57.021428 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8a04af6c-fb4a-48e5-b2d7-f3d54cab5aa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '720004e5661c4843aa6f016ce41ff76a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:57.022651 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8a04af6c-fb4a-48e5-b2d7-f3d54cab5aa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '720004e5661c4843aa6f016ce41ff76a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:57.047782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b17b16-06f4-4c3f-9752-982b122cf495 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:57.063045 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d54cf3-77b9-45de-ba20-d76da5552de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:57.064031 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d54cf3-77b9-45de-ba20-d76da5552de1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:57.064308 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d54cf3-77b9-45de-ba20-d76da5552de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:57.064589 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d54cf3-77b9-45de-ba20-d76da5552de1 admin admin] PATH_INFO: `/v3/projects/e58f0222b5494864b39bb2eeedcaf28a/users/6f5e3f56452e4b6e8f92277f0484f0d3/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:57.092501 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 867/1726] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:24:56 2022] DELETE /identity/v3/users/720004e5661c4843aa6f016ce41ff76a => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:24:57.100301 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979ba4a-f28f-48c9-b9b3-596293ae29ad None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:57.100776 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979ba4a-f28f-48c9-b9b3-596293ae29ad None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:57.101282 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979ba4a-f28f-48c9-b9b3-596293ae29ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:57.138622 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d54cf3-77b9-45de-ba20-d76da5552de1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e58f0222b5494864b39bb2eeedcaf28a, user_id=6f5e3f56452e4b6e8f92277f0484f0d3, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:57.147075 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d54cf3-77b9-45de-ba20-d76da5552de1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:57.203490 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 860/1727] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:24:57 2022] PUT /identity/v3/projects/e58f0222b5494864b39bb2eeedcaf28a/users/6f5e3f56452e4b6e8f92277f0484f0d3/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:24:57.211673 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a9d29-64ae-414a-ba3c-1cbc16cd3951 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:57.212125 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a9d29-64ae-414a-ba3c-1cbc16cd3951 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:57.212525 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a9d29-64ae-414a-ba3c-1cbc16cd3951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:57.240411 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1979ba4a-f28f-48c9-b9b3-596293ae29ad None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:57.248002 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1979ba4a-f28f-48c9-b9b3-596293ae29ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:57.333732 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-268a9d29-64ae-414a-ba3c-1cbc16cd3951 None None] MFA Rules not processed for user `6f5e3f56452e4b6e8f92277f0484f0d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:57.335819 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-268a9d29-64ae-414a-ba3c-1cbc16cd3951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:57.646034 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 868/1728] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 548 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:57.663040 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979ba4a-f28f-48c9-b9b3-596293ae29ad None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:57.675453 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1979ba4a-f28f-48c9-b9b3-596293ae29ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:57.682294 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 861/1729] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:57 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:57.684041 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a9d29-64ae-414a-ba3c-1cbc16cd3951 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:57.689430 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-268a9d29-64ae-414a-ba3c-1cbc16cd3951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:57.778413 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a9d29-64ae-414a-ba3c-1cbc16cd3951 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:57.792901 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979ba4a-f28f-48c9-b9b3-596293ae29ad None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:57.808290 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f213e411-63a2-440b-85e5-b2fe6d591a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:57.809523 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f213e411-63a2-440b-85e5-b2fe6d591a60 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:57.810001 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f213e411-63a2-440b-85e5-b2fe6d591a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:57.810492 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f213e411-63a2-440b-85e5-b2fe6d591a60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:57.811822 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f213e411-63a2-440b-85e5-b2fe6d591a60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:57.819811 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bef28e-795b-4414-b74b-5ea6dc31364d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:57.821826 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bef28e-795b-4414-b74b-5ea6dc31364d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:57.822842 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bef28e-795b-4414-b74b-5ea6dc31364d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:57.823177 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f213e411-63a2-440b-85e5-b2fe6d591a60 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:57.823769 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bef28e-795b-4414-b74b-5ea6dc31364d admin admin] PATH_INFO: `/v3/projects/efbdd83c95884ccda10a6c0b43d8c09c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:57.830005 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bef28e-795b-4414-b74b-5ea6dc31364d admin admin] RBAC: Authorizing `identity:delete_project(project_id=efbdd83c95884ccda10a6c0b43d8c09c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:57.835304 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bef28e-795b-4414-b74b-5ea6dc31364d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:57.951043 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:57.998103 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 862/1730] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:24:57 2022] POST /identity/v3/projects => generated 363 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:24:58.004821 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f213e411-63a2-440b-85e5-b2fe6d591a60 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:58.086600 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:58.203798 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e0bef28e-795b-4414-b74b-5ea6dc31364d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efbdd83c95884ccda10a6c0b43d8c09c', 'request_id': 'req-e0bef28e-795b-4414-b74b-5ea6dc31364d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:58.269837 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e0bef28e-795b-4414-b74b-5ea6dc31364d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efbdd83c95884ccda10a6c0b43d8c09c', 'request_id': 'req-e0bef28e-795b-4414-b74b-5ea6dc31364d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:58.309673 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f213e411-63a2-440b-85e5-b2fe6d591a60 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:58.323309 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e0bef28e-795b-4414-b74b-5ea6dc31364d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efbdd83c95884ccda10a6c0b43d8c09c', 'request_id': 'req-e0bef28e-795b-4414-b74b-5ea6dc31364d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:58.390195 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e0bef28e-795b-4414-b74b-5ea6dc31364d admin admin] The token cache is being invalidate because project efbdd83c95884ccda10a6c0b43d8c09c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:24:58.390794 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e0bef28e-795b-4414-b74b-5ea6dc31364d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:58.394771 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e0bef28e-795b-4414-b74b-5ea6dc31364d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:24:58.402834 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 869/1731] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:24:57 2022] DELETE /identity/v3/projects/efbdd83c95884ccda10a6c0b43d8c09c => generated 0 bytes in 756 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:24:58.404786 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bef28e-795b-4414-b74b-5ea6dc31364d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:58.415364 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e0bef28e-795b-4414-b74b-5ea6dc31364d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:58.604112 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24fb851-c889-4a9c-abf0-1ffb3bca228f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:58.607259 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24fb851-c889-4a9c-abf0-1ffb3bca228f admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:58.608383 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24fb851-c889-4a9c-abf0-1ffb3bca228f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:58.608383 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24fb851-c889-4a9c-abf0-1ffb3bca228f admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:58.615340 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24fb851-c889-4a9c-abf0-1ffb3bca228f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:58.630177 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24fb851-c889-4a9c-abf0-1ffb3bca228f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:58.664750 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bef28e-795b-4414-b74b-5ea6dc31364d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:58.687042 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f51d1e5-dab0-4449-acf6-d5a2341019f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:58.688540 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51d1e5-dab0-4449-acf6-d5a2341019f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:58.689104 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51d1e5-dab0-4449-acf6-d5a2341019f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:58.689770 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51d1e5-dab0-4449-acf6-d5a2341019f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:58.693515 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 863/1732] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:24:58 2022] POST /identity/v3/users => generated 288 bytes in 690 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:24:58.699063 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0f51d1e5-dab0-4449-acf6-d5a2341019f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:58.700968 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050af73-75da-418e-ad58-fcac5fd2d767 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:58.701405 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050af73-75da-418e-ad58-fcac5fd2d767 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:58.701802 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050af73-75da-418e-ad58-fcac5fd2d767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:58.756812 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f51d1e5-dab0-4449-acf6-d5a2341019f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:58.757218 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1050af73-75da-418e-ad58-fcac5fd2d767 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:58.759045 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1050af73-75da-418e-ad58-fcac5fd2d767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:58.761210 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f51d1e5-dab0-4449-acf6-d5a2341019f6 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:58.802760 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 870/1733] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:24:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:24:58.804125 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f51d1e5-dab0-4449-acf6-d5a2341019f6 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:58.811267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0f51d1e5-dab0-4449-acf6-d5a2341019f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:58.822840 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 864/1734] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:58.843046 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27897e3c-3a45-4877-ab37-b74c5d54983c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:58.844320 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27897e3c-3a45-4877-ab37-b74c5d54983c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:58.845031 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27897e3c-3a45-4877-ab37-b74c5d54983c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:58.903102 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f51d1e5-dab0-4449-acf6-d5a2341019f6 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:58.925402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-27897e3c-3a45-4877-ab37-b74c5d54983c None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:58.933167 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-27897e3c-3a45-4877-ab37-b74c5d54983c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:58.960535 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f7efb-06d2-4d3c-bd59-b9cd385643d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:58.967262 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f7efb-06d2-4d3c-bd59-b9cd385643d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:58.968851 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f7efb-06d2-4d3c-bd59-b9cd385643d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:58.969344 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f7efb-06d2-4d3c-bd59-b9cd385643d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:58.970703 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f7efb-06d2-4d3c-bd59-b9cd385643d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:58.976332 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 865/1735] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:24:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:58.980975 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27897e3c-3a45-4877-ab37-b74c5d54983c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:59.004711 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f7efb-06d2-4d3c-bd59-b9cd385643d5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:59.039354 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27897e3c-3a45-4877-ab37-b74c5d54983c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:59.071234 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca247e88-60af-4f1c-b22d-719091f1292a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:59.071234 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca247e88-60af-4f1c-b22d-719091f1292a admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:59.071234 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca247e88-60af-4f1c-b22d-719091f1292a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:59.071234 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca247e88-60af-4f1c-b22d-719091f1292a admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:59.074387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 871/1736] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:24:59.075371 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca247e88-60af-4f1c-b22d-719091f1292a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:24:59.075371 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca247e88-60af-4f1c-b22d-719091f1292a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:59.077548 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca247e88-60af-4f1c-b22d-719091f1292a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:59.081658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f7efb-06d2-4d3c-bd59-b9cd385643d5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:59.130480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f7efb-06d2-4d3c-bd59-b9cd385643d5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:59.132315 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 866/1737] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:24:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:24:59.139213 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca247e88-60af-4f1c-b22d-719091f1292a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:59.177777 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac152ee-b36c-4fc5-8dbb-9eab8f39f3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:59.182216 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac152ee-b36c-4fc5-8dbb-9eab8f39f3b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:59.184308 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac152ee-b36c-4fc5-8dbb-9eab8f39f3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:59.184796 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac152ee-b36c-4fc5-8dbb-9eab8f39f3b3 admin admin] PATH_INFO: `/v3/projects/f4bcb443537541a4b35b237a3f206c05/users/4e2d1bc2fe564267a0675bcfb5ffb7ad/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:59.189128 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca247e88-60af-4f1c-b22d-719091f1292a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:59.226286 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c05d45a-f405-49f5-9e65-0a4ef7737885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:59.237194 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c05d45a-f405-49f5-9e65-0a4ef7737885 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:59.238032 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c05d45a-f405-49f5-9e65-0a4ef7737885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:59.238806 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c05d45a-f405-49f5-9e65-0a4ef7737885 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:59.240244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c05d45a-f405-49f5-9e65-0a4ef7737885 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:59.244235 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c05d45a-f405-49f5-9e65-0a4ef7737885 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:59.294716 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac152ee-b36c-4fc5-8dbb-9eab8f39f3b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4bcb443537541a4b35b237a3f206c05, user_id=4e2d1bc2fe564267a0675bcfb5ffb7ad, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:59.306866 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac152ee-b36c-4fc5-8dbb-9eab8f39f3b3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:59.325257 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 867/1738] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:24:59 2022] POST /identity/v3/projects => generated 373 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:24:59.325257 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396fd851-7b0e-49db-8940-4aed30ebd0f7 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:59.325257 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396fd851-7b0e-49db-8940-4aed30ebd0f7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:59.325257 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396fd851-7b0e-49db-8940-4aed30ebd0f7 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:59.332592 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 868/1739] 10.222.0.18 () {54 vars in 961 bytes} [Mon Jul 25 19:24:59 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:24:59.334593 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396fd851-7b0e-49db-8940-4aed30ebd0f7 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:59.373743 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 872/1740] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:24:59 2022] PUT /identity/v3/projects/f4bcb443537541a4b35b237a3f206c05/users/4e2d1bc2fe564267a0675bcfb5ffb7ad/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:24:59.378496 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2369a304-f0a5-4385-9059-7317a65f956b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:59.379314 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2369a304-f0a5-4385-9059-7317a65f956b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:59.379887 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2369a304-f0a5-4385-9059-7317a65f956b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:59.473334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2369a304-f0a5-4385-9059-7317a65f956b None None] MFA Rules not processed for user `0422d6695bf14925bf0f993cb16e64ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:24:59.475078 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2369a304-f0a5-4385-9059-7317a65f956b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:24:59.648997 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396fd851-7b0e-49db-8940-4aed30ebd0f7 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:59.671205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7b5268-91ae-4609-92d9-5630517a449a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:59.672640 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b5268-91ae-4609-92d9-5630517a449a admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:59.673133 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b5268-91ae-4609-92d9-5630517a449a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:59.673494 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b5268-91ae-4609-92d9-5630517a449a admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:59.674668 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7b5268-91ae-4609-92d9-5630517a449a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:59.678609 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7b5268-91ae-4609-92d9-5630517a449a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:59.717678 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:59.770580 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 869/1741] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:24:59 2022] POST /identity/v3/users => generated 287 bytes in 435 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:24:59.773308 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7b5268-91ae-4609-92d9-5630517a449a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:59.793498 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 873/1742] 10.222.0.18 () {58 vars in 1058 bytes} [Mon Jul 25 19:24:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:24:59.803968 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2369a304-f0a5-4385-9059-7317a65f956b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:59.816178 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:24:59.842479 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2369a304-f0a5-4385-9059-7317a65f956b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:59.857509 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7b5268-91ae-4609-92d9-5630517a449a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:59.872580 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7526553-2a58-4e51-af30-6d763fcdf637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:59.875750 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7526553-2a58-4e51-af30-6d763fcdf637 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:59.881482 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7526553-2a58-4e51-af30-6d763fcdf637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:59.882268 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7526553-2a58-4e51-af30-6d763fcdf637 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:59.885026 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7526553-2a58-4e51-af30-6d763fcdf637 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:59.891404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7526553-2a58-4e51-af30-6d763fcdf637 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:59.892891 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6c9a4-38c9-4d51-a104-d6837034f14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:59.894386 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6c9a4-38c9-4d51-a104-d6837034f14e admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:24:59.897670 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6c9a4-38c9-4d51-a104-d6837034f14e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:24:59.898562 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6c9a4-38c9-4d51-a104-d6837034f14e admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:24:59.900329 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f6c9a4-38c9-4d51-a104-d6837034f14e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:24:59.905771 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f6c9a4-38c9-4d51-a104-d6837034f14e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:24:59.941642 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 874/1743] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:24:59.943328 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7526553-2a58-4e51-af30-6d763fcdf637 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:59.956832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 870/1744] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:24:59.956832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6c9a4-38c9-4d51-a104-d6837034f14e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:24:59.984978 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7526553-2a58-4e51-af30-6d763fcdf637 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:24:59.997547 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f25c939-344a-4fce-9fb8-d4a764f3f2c1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0422d6695bf14925bf0f993cb16e64ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:24:59.999576 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25c939-344a-4fce-9fb8-d4a764f3f2c1 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:00.000173 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6c9a4-38c9-4d51-a104-d6837034f14e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:00.000566 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25c939-344a-4fce-9fb8-d4a764f3f2c1 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:00.001938 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25c939-344a-4fce-9fb8-d4a764f3f2c1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:00.007339 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4f25c939-344a-4fce-9fb8-d4a764f3f2c1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:00.040314 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a4aee-c894-40d0-9bb3-eccc5d773069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:00.041507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a4aee-c894-40d0-9bb3-eccc5d773069 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:00.042506 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a4aee-c894-40d0-9bb3-eccc5d773069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:00.043006 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a4aee-c894-40d0-9bb3-eccc5d773069 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:00.060159 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a4aee-c894-40d0-9bb3-eccc5d773069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:00.063767 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a4aee-c894-40d0-9bb3-eccc5d773069 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:00.081383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f25c939-344a-4fce-9fb8-d4a764f3f2c1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:00.094157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f25c939-344a-4fce-9fb8-d4a764f3f2c1 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:00.146008 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 871/1745] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:25:00.146008 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a4aee-c894-40d0-9bb3-eccc5d773069 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:00.171356 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 875/1746] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:24:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:25:00.171356 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f25c939-344a-4fce-9fb8-d4a764f3f2c1 service placement] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:00.193939 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a4aee-c894-40d0-9bb3-eccc5d773069 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:00.215679 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf36d48-8615-4d62-9053-e38f7154fd25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:00.216135 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f25c939-344a-4fce-9fb8-d4a764f3f2c1 service placement] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:00.217363 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf36d48-8615-4d62-9053-e38f7154fd25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:00.221518 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf36d48-8615-4d62-9053-e38f7154fd25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:00.221518 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf36d48-8615-4d62-9053-e38f7154fd25 admin admin] PATH_INFO: `/v3/projects/f4bcb443537541a4b35b237a3f206c05/users/4e2d1bc2fe564267a0675bcfb5ffb7ad/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:00.233769 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf36d48-8615-4d62-9053-e38f7154fd25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4bcb443537541a4b35b237a3f206c05, user_id=4e2d1bc2fe564267a0675bcfb5ffb7ad, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:00.238438 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba5c98-ad28-4b11-a298-95086df99fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:00.240115 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba5c98-ad28-4b11-a298-95086df99fb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:00.240650 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba5c98-ad28-4b11-a298-95086df99fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:00.241252 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba5c98-ad28-4b11-a298-95086df99fb0 admin admin] PATH_INFO: `/v3/projects/25243ec36d0f49f985fb689734d298df/users/fd0059d35acc476cb9962adc6ae8ec34/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:00.246606 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf36d48-8615-4d62-9053-e38f7154fd25 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:00.326218 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba5c98-ad28-4b11-a298-95086df99fb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25243ec36d0f49f985fb689734d298df, user_id=fd0059d35acc476cb9962adc6ae8ec34, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:00.349450 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 872/1747] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:25:00 2022] PUT /identity/v3/projects/f4bcb443537541a4b35b237a3f206c05/users/4e2d1bc2fe564267a0675bcfb5ffb7ad/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:00.349745 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba5c98-ad28-4b11-a298-95086df99fb0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:00.361871 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff6305-622b-4269-8cef-6436a89ae8e3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:00.362411 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff6305-622b-4269-8cef-6436a89ae8e3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:00.362871 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff6305-622b-4269-8cef-6436a89ae8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:00.412762 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 876/1748] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:25:00 2022] PUT /identity/v3/projects/25243ec36d0f49f985fb689734d298df/users/fd0059d35acc476cb9962adc6ae8ec34/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:00.440721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c92b0-cc4c-4062-9ff9-487ddc6316fe None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:00.441200 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c92b0-cc4c-4062-9ff9-487ddc6316fe None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:00.441701 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c92b0-cc4c-4062-9ff9-487ddc6316fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:00.489125 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-22ff6305-622b-4269-8cef-6436a89ae8e3 None None] MFA Rules not processed for user `4e2d1bc2fe564267a0675bcfb5ffb7ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:00.493782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-22ff6305-622b-4269-8cef-6436a89ae8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:00.569966 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2c0c92b0-cc4c-4062-9ff9-487ddc6316fe None None] MFA Rules not processed for user `fd0059d35acc476cb9962adc6ae8ec34`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:00.572380 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2c0c92b0-cc4c-4062-9ff9-487ddc6316fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:00.847439 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 873/1749] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:00 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 488 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:25:00.853624 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2e8a1-c390-47d6-af3a-ad5dd78d7e38 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:00.854436 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2e8a1-c390-47d6-af3a-ad5dd78d7e38 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:00.855257 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2e8a1-c390-47d6-af3a-ad5dd78d7e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:00.939452 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 877/1750] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 502 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:25:00.941048 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0c92b0-cc4c-4062-9ff9-487ddc6316fe None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:00.953869 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2c0c92b0-cc4c-4062-9ff9-487ddc6316fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:00.967075 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-fdd2e8a1-c390-47d6-af3a-ad5dd78d7e38 None None] MFA Rules not processed for user `6ecf34ceee294b53b5be7c1723b42950`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:00.969165 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fdd2e8a1-c390-47d6-af3a-ad5dd78d7e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:01.274433 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0c92b0-cc4c-4062-9ff9-487ddc6316fe None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:01.289953 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf5181d-693c-401e-b271-e7ae14950989 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:01.290988 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf5181d-693c-401e-b271-e7ae14950989 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:01.292032 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf5181d-693c-401e-b271-e7ae14950989 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:01.292509 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf5181d-693c-401e-b271-e7ae14950989 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:01.297004 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 874/1751] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 446 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:25:01.298981 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2e8a1-c390-47d6-af3a-ad5dd78d7e38 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:01.336077 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf5181d-693c-401e-b271-e7ae14950989 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:01.338931 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf5181d-693c-401e-b271-e7ae14950989 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:01.389898 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 878/1752] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:25:00 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:25:01.391545 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf5181d-693c-401e-b271-e7ae14950989 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:01.399039 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-edf5181d-693c-401e-b271-e7ae14950989 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:01.444141 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:01.618976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2e8a1-c390-47d6-af3a-ad5dd78d7e38 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:01.643802 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:01.691818 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac309e-c0a7-4520-a4e8-439db37d46a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:01.693061 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac309e-c0a7-4520-a4e8-439db37d46a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:01.696489 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac309e-c0a7-4520-a4e8-439db37d46a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:01.696891 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac309e-c0a7-4520-a4e8-439db37d46a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:01.702533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac309e-c0a7-4520-a4e8-439db37d46a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:01.706871 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac309e-c0a7-4520-a4e8-439db37d46a3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:01.826927 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf5181d-693c-401e-b271-e7ae14950989 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:01.855624 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 875/1753] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:25:01 2022] POST /identity/v3/projects => generated 377 bytes in 554 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:25:01.858777 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac309e-c0a7-4520-a4e8-439db37d46a3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:01.870675 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-aeac309e-c0a7-4520-a4e8-439db37d46a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:02.133638 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97528c0d-5197-4c36-89e0-b74b9c8bd48e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:02.135320 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97528c0d-5197-4c36-89e0-b74b9c8bd48e service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:02.137935 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97528c0d-5197-4c36-89e0-b74b9c8bd48e service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:02.142913 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97528c0d-5197-4c36-89e0-b74b9c8bd48e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:02.240199 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac309e-c0a7-4520-a4e8-439db37d46a3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:02.282649 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d36d16-7d5a-4c12-bfa4-86a48554fb33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:02.288493 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d36d16-7d5a-4c12-bfa4-86a48554fb33 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:02.290181 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d36d16-7d5a-4c12-bfa4-86a48554fb33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:02.290710 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d36d16-7d5a-4c12-bfa4-86a48554fb33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:02.306327 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b6d36d16-7d5a-4c12-bfa4-86a48554fb33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:02.661960 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97528c0d-5197-4c36-89e0-b74b9c8bd48e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:02.669731 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97528c0d-5197-4c36-89e0-b74b9c8bd48e service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:02.673007 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d36d16-7d5a-4c12-bfa4-86a48554fb33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:02.677316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d36d16-7d5a-4c12-bfa4-86a48554fb33 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:02.706483 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 879/1754] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:25:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 1314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:25:02.716407 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97528c0d-5197-4c36-89e0-b74b9c8bd48e service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:02.743275 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 876/1755] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:25:01 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 888 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:25:02.974965 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97528c0d-5197-4c36-89e0-b74b9c8bd48e service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:02.999502 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593f672-a283-4d55-8c6a-08f709465de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:03.000520 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593f672-a283-4d55-8c6a-08f709465de1 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:03.001004 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593f672-a283-4d55-8c6a-08f709465de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:03.001421 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593f672-a283-4d55-8c6a-08f709465de1 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:03.003378 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593f672-a283-4d55-8c6a-08f709465de1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:03.019249 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593f672-a283-4d55-8c6a-08f709465de1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:03.100253 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 880/1756] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:25:02 2022] POST /identity/v3/users => generated 295 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:25:03.107925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d36d16-7d5a-4c12-bfa4-86a48554fb33 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:03.185647 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d36d16-7d5a-4c12-bfa4-86a48554fb33 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:03.208709 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf88096-6ca3-4be0-b4fa-6e490055cdda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:03.209832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf88096-6ca3-4be0-b4fa-6e490055cdda admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:03.210327 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf88096-6ca3-4be0-b4fa-6e490055cdda admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:03.210824 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf88096-6ca3-4be0-b4fa-6e490055cdda admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:03.215112 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf88096-6ca3-4be0-b4fa-6e490055cdda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:03.226065 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf88096-6ca3-4be0-b4fa-6e490055cdda admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:03.302762 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 877/1757] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:25:03.314158 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593f672-a283-4d55-8c6a-08f709465de1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:03.355714 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593f672-a283-4d55-8c6a-08f709465de1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:03.380536 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afbce5a-4053-471e-85ea-240687733564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:03.382289 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbce5a-4053-471e-85ea-240687733564 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:03.382808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbce5a-4053-471e-85ea-240687733564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:03.383311 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbce5a-4053-471e-85ea-240687733564 admin admin] PATH_INFO: `/v3/projects/22a61edbd6f34cb7935555d9b0e45367/users/7d112b8e69e24c1fa8a7dc6bd3d4dadf/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:03.421952 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:03.455564 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afbce5a-4053-471e-85ea-240687733564 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22a61edbd6f34cb7935555d9b0e45367, user_id=7d112b8e69e24c1fa8a7dc6bd3d4dadf, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:03.468043 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afbce5a-4053-471e-85ea-240687733564 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:03.536386 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 881/1758] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:25:03 2022] PUT /identity/v3/projects/22a61edbd6f34cb7935555d9b0e45367/users/7d112b8e69e24c1fa8a7dc6bd3d4dadf/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:03.543306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf88096-6ca3-4be0-b4fa-6e490055cdda admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:03.580725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf88096-6ca3-4be0-b4fa-6e490055cdda admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:03.829884 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:03.862241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1f5675-9ed2-42f8-80e4-a59cc09024ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:03.867295 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f5675-9ed2-42f8-80e4-a59cc09024ac admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:03.867841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f5675-9ed2-42f8-80e4-a59cc09024ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:03.871384 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f5675-9ed2-42f8-80e4-a59cc09024ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:03.875353 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1f5675-9ed2-42f8-80e4-a59cc09024ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:03.886014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1f5675-9ed2-42f8-80e4-a59cc09024ac admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:03.917195 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afbce5a-4053-471e-85ea-240687733564 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:03.921307 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6afbce5a-4053-471e-85ea-240687733564 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:03.971823 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 878/1759] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:25:03.974566 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1f5675-9ed2-42f8-80e4-a59cc09024ac admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:04.000139 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afbce5a-4053-471e-85ea-240687733564 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:04.020970 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1f5675-9ed2-42f8-80e4-a59cc09024ac admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:04.022867 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e4fde-e53e-40dd-b44b-c36a944cee62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:04.024383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e4fde-e53e-40dd-b44b-c36a944cee62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:04.026750 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e4fde-e53e-40dd-b44b-c36a944cee62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:04.026750 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e4fde-e53e-40dd-b44b-c36a944cee62 admin admin] PATH_INFO: `/v3/projects/498b69be7a894313aaf5016790b30d4c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:04.032024 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2916227f-95b4-4b55-b95e-7edd6e9dbda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:04.033314 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370e4fde-e53e-40dd-b44b-c36a944cee62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=498b69be7a894313aaf5016790b30d4c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:04.033845 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916227f-95b4-4b55-b95e-7edd6e9dbda1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:04.034345 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916227f-95b4-4b55-b95e-7edd6e9dbda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:04.035061 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916227f-95b4-4b55-b95e-7edd6e9dbda1 admin admin] PATH_INFO: `/v3/projects/22a61edbd6f34cb7935555d9b0e45367/users/7d112b8e69e24c1fa8a7dc6bd3d4dadf/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:04.039341 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2916227f-95b4-4b55-b95e-7edd6e9dbda1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22a61edbd6f34cb7935555d9b0e45367, user_id=7d112b8e69e24c1fa8a7dc6bd3d4dadf, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:04.042739 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370e4fde-e53e-40dd-b44b-c36a944cee62 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:04.046125 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2916227f-95b4-4b55-b95e-7edd6e9dbda1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:04.079635 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 879/1760] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:25:03 2022] PUT /identity/v3/projects/22a61edbd6f34cb7935555d9b0e45367/users/7d112b8e69e24c1fa8a7dc6bd3d4dadf/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:04.101208 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2b696-9d5b-402f-95ee-d36258cfbd45 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:04.102324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2b696-9d5b-402f-95ee-d36258cfbd45 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:04.102983 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2b696-9d5b-402f-95ee-d36258cfbd45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:04.236940 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-34e2b696-9d5b-402f-95ee-d36258cfbd45 None None] MFA Rules not processed for user `7d112b8e69e24c1fa8a7dc6bd3d4dadf`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:04.277304 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-34e2b696-9d5b-402f-95ee-d36258cfbd45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:04.301107 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-370e4fde-e53e-40dd-b44b-c36a944cee62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '498b69be7a894313aaf5016790b30d4c', 'request_id': 'req-370e4fde-e53e-40dd-b44b-c36a944cee62'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:04.334446 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-370e4fde-e53e-40dd-b44b-c36a944cee62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '498b69be7a894313aaf5016790b30d4c', 'request_id': 'req-370e4fde-e53e-40dd-b44b-c36a944cee62'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:04.348558 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-370e4fde-e53e-40dd-b44b-c36a944cee62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '498b69be7a894313aaf5016790b30d4c', 'request_id': 'req-370e4fde-e53e-40dd-b44b-c36a944cee62'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:04.389312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-370e4fde-e53e-40dd-b44b-c36a944cee62 admin admin] The token cache is being invalidate because project 498b69be7a894313aaf5016790b30d4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:25:04.390651 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-370e4fde-e53e-40dd-b44b-c36a944cee62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:04.398367 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-370e4fde-e53e-40dd-b44b-c36a944cee62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:04.409812 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 882/1761] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:25:03 2022] DELETE /identity/v3/projects/498b69be7a894313aaf5016790b30d4c => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:25:04.413805 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e4fde-e53e-40dd-b44b-c36a944cee62 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:04.418646 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-370e4fde-e53e-40dd-b44b-c36a944cee62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:04.720481 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e4fde-e53e-40dd-b44b-c36a944cee62 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:04.741317 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20758b6e-c6fe-455e-97e3-d8a37ced08d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:04.742841 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20758b6e-c6fe-455e-97e3-d8a37ced08d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:04.744590 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20758b6e-c6fe-455e-97e3-d8a37ced08d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:04.745676 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20758b6e-c6fe-455e-97e3-d8a37ced08d0 admin admin] PATH_INFO: `/v3/users/d2eb8262d2164fdf8303f443439689ac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:04.747799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20758b6e-c6fe-455e-97e3-d8a37ced08d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2eb8262d2164fdf8303f443439689ac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:04.752010 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20758b6e-c6fe-455e-97e3-d8a37ced08d0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:04.808470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 880/1762] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:04 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 711 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:25:04.854159 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e2b696-9d5b-402f-95ee-d36258cfbd45 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:04.857397 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-34e2b696-9d5b-402f-95ee-d36258cfbd45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:04.968567 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-20758b6e-c6fe-455e-97e3-d8a37ced08d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2eb8262d2164fdf8303f443439689ac'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:04.993196 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-20758b6e-c6fe-455e-97e3-d8a37ced08d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2eb8262d2164fdf8303f443439689ac'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:05.012278 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-20758b6e-c6fe-455e-97e3-d8a37ced08d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2eb8262d2164fdf8303f443439689ac'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:05.015832 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-20758b6e-c6fe-455e-97e3-d8a37ced08d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2eb8262d2164fdf8303f443439689ac'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:05.019265 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-20758b6e-c6fe-455e-97e3-d8a37ced08d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2eb8262d2164fdf8303f443439689ac'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:05.040444 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=81,keystone:INSERT=3,keystone:DELETE=16 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:05.137164 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 883/1763] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:25:04 2022] DELETE /identity/v3/users/d2eb8262d2164fdf8303f443439689ac => generated 0 bytes in 726 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:05.242870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e2b696-9d5b-402f-95ee-d36258cfbd45 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:05.317550 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e909a-5492-4c98-ab27-0dc97359e4d0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:05.317992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e909a-5492-4c98-ab27-0dc97359e4d0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:05.318475 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e909a-5492-4c98-ab27-0dc97359e4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:05.465355 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c18e909a-5492-4c98-ab27-0dc97359e4d0 None None] MFA Rules not processed for user `0ed3ed6183d34c73ba40d13af037f838`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:05.469280 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c18e909a-5492-4c98-ab27-0dc97359e4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:05.658235 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b453ca-9e7d-4d1c-9e66-52c16c45c122 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:05.662996 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b453ca-9e7d-4d1c-9e66-52c16c45c122 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:05.663738 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b453ca-9e7d-4d1c-9e66-52c16c45c122 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:05.664171 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b453ca-9e7d-4d1c-9e66-52c16c45c122 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:05.670065 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-28b453ca-9e7d-4d1c-9e66-52c16c45c122 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:05.772153 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 884/1764] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:25:05.786756 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59622013-9488-4a5c-a8dd-9a097b57565d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:05.787464 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59622013-9488-4a5c-a8dd-9a097b57565d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:05.788152 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59622013-9488-4a5c-a8dd-9a097b57565d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:05.819312 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:05.856796 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-59622013-9488-4a5c-a8dd-9a097b57565d None None] MFA Rules not processed for user `0ed3ed6183d34c73ba40d13af037f838`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:05.858269 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-59622013-9488-4a5c-a8dd-9a097b57565d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:05.881640 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 885/1765] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:25:05.905022 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59622013-9488-4a5c-a8dd-9a097b57565d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:05.910950 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-59622013-9488-4a5c-a8dd-9a097b57565d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:05.933344 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b453ca-9e7d-4d1c-9e66-52c16c45c122 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:05.936715 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b453ca-9e7d-4d1c-9e66-52c16c45c122 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:05.960525 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 881/1766] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:25:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 1108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:25:06.241574 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59622013-9488-4a5c-a8dd-9a097b57565d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:06.256612 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ae5d0-1ebd-40df-a93e-6ddb98e5529b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:06.257979 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ae5d0-1ebd-40df-a93e-6ddb98e5529b service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:06.258569 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ae5d0-1ebd-40df-a93e-6ddb98e5529b service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:06.259536 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ae5d0-1ebd-40df-a93e-6ddb98e5529b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:06.321022 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ae5d0-1ebd-40df-a93e-6ddb98e5529b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:06.335129 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ae5d0-1ebd-40df-a93e-6ddb98e5529b service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:06.372038 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 886/1767] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:25:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:25:06.649387 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9669839-c724-422e-94b2-979c2fd380f8 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:06.649688 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9669839-c724-422e-94b2-979c2fd380f8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:06.649980 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9669839-c724-422e-94b2-979c2fd380f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:06.743354 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e9669839-c724-422e-94b2-979c2fd380f8 None None] MFA Rules not processed for user `1c0772205ae24713b8901666a87018b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:06.746183 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e9669839-c724-422e-94b2-979c2fd380f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:06.962641 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 882/1768] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Jul 25 19:25:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:25:07.355294 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ae5d0-1ebd-40df-a93e-6ddb98e5529b service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:07.358809 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d62ae5d0-1ebd-40df-a93e-6ddb98e5529b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:07.607383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:07.745806 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ae5d0-1ebd-40df-a93e-6ddb98e5529b service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:07.757015 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c708515-60b5-4203-8f55-f6aa2e236aa7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feafcf1b5a9844fe9738823fddd07e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:07.758746 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c708515-60b5-4203-8f55-f6aa2e236aa7 service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:07.759749 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c708515-60b5-4203-8f55-f6aa2e236aa7 service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:07.760480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c708515-60b5-4203-8f55-f6aa2e236aa7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:07.786983 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c708515-60b5-4203-8f55-f6aa2e236aa7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:07.791058 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c708515-60b5-4203-8f55-f6aa2e236aa7 service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:07.838709 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 887/1769] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Jul 25 19:25:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 19:25:08.976386 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9669839-c724-422e-94b2-979c2fd380f8 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:08.982254 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e9669839-c724-422e-94b2-979c2fd380f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:09.111533 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45496e7d-e4a6-4ecc-a74e-c5bb2a21bf96 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:09.112328 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45496e7d-e4a6-4ecc-a74e-c5bb2a21bf96 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:09.112586 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45496e7d-e4a6-4ecc-a74e-c5bb2a21bf96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:09.363058 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-45496e7d-e4a6-4ecc-a74e-c5bb2a21bf96 None None] MFA Rules not processed for user `30efc9bc66ce498d9d94980d85152694`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:09.365839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-45496e7d-e4a6-4ecc-a74e-c5bb2a21bf96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:09.460600 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9669839-c724-422e-94b2-979c2fd380f8 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:09.500132 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda0221-3d0d-495a-9e66-5036fb5a7b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:09.501997 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda0221-3d0d-495a-9e66-5036fb5a7b07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:09.501997 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda0221-3d0d-495a-9e66-5036fb5a7b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:09.502334 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda0221-3d0d-495a-9e66-5036fb5a7b07 admin admin] PATH_INFO: `/v3/users/6f5e3f56452e4b6e8f92277f0484f0d3` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:09.519850 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda0221-3d0d-495a-9e66-5036fb5a7b07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f5e3f56452e4b6e8f92277f0484f0d3)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:09.533019 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda0221-3d0d-495a-9e66-5036fb5a7b07 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:09.841295 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1eda0221-3d0d-495a-9e66-5036fb5a7b07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f5e3f56452e4b6e8f92277f0484f0d3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:09.858103 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:09.865042 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1eda0221-3d0d-495a-9e66-5036fb5a7b07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f5e3f56452e4b6e8f92277f0484f0d3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:09.905167 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 888/1770] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 796 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:25:09.907501 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45496e7d-e4a6-4ecc-a74e-c5bb2a21bf96 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:09.919582 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-45496e7d-e4a6-4ecc-a74e-c5bb2a21bf96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:09.924263 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1eda0221-3d0d-495a-9e66-5036fb5a7b07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f5e3f56452e4b6e8f92277f0484f0d3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:09.925263 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1eda0221-3d0d-495a-9e66-5036fb5a7b07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f5e3f56452e4b6e8f92277f0484f0d3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:09.973836 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1eda0221-3d0d-495a-9e66-5036fb5a7b07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f5e3f56452e4b6e8f92277f0484f0d3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:09.996420 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 883/1771] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:25:08 2022] DELETE /identity/v3/users/6f5e3f56452e4b6e8f92277f0484f0d3 => generated 0 bytes in 1019 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:09.996420 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3d722-4ab2-4fe1-96f1-515238cb39b8 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:09.996420 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3d722-4ab2-4fe1-96f1-515238cb39b8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:09.996420 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3d722-4ab2-4fe1-96f1-515238cb39b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:10.048843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-cfd3d722-4ab2-4fe1-96f1-515238cb39b8 None None] MFA Rules not processed for user `30efc9bc66ce498d9d94980d85152694`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:10.050527 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cfd3d722-4ab2-4fe1-96f1-515238cb39b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:10.059051 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45496e7d-e4a6-4ecc-a74e-c5bb2a21bf96 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:10.317674 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5737de09-3547-4d64-be72-8bb7e6cc2a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:10.322244 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737de09-3547-4d64-be72-8bb7e6cc2a60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:10.328280 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737de09-3547-4d64-be72-8bb7e6cc2a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:10.328280 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737de09-3547-4d64-be72-8bb7e6cc2a60 admin admin] PATH_INFO: `/v3/projects/19f9bf95e41f439c8956f250859575f0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:10.335217 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5737de09-3547-4d64-be72-8bb7e6cc2a60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19f9bf95e41f439c8956f250859575f0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:10.344253 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 884/1772] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:25:10.350298 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe818c-4fc5-4dca-a3c4-c89aae631ccc None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:10.365060 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5737de09-3547-4d64-be72-8bb7e6cc2a60 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:10.365977 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe818c-4fc5-4dca-a3c4-c89aae631ccc None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:10.366682 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe818c-4fc5-4dca-a3c4-c89aae631ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:10.458033 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ccfe818c-4fc5-4dca-a3c4-c89aae631ccc None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:10.460717 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ccfe818c-4fc5-4dca-a3c4-c89aae631ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:10.488749 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 885/1773] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:25:10.490948 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe818c-4fc5-4dca-a3c4-c89aae631ccc None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:10.496369 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ccfe818c-4fc5-4dca-a3c4-c89aae631ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:10.676474 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5737de09-3547-4d64-be72-8bb7e6cc2a60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19f9bf95e41f439c8956f250859575f0', 'request_id': 'req-5737de09-3547-4d64-be72-8bb7e6cc2a60'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:10.706742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5737de09-3547-4d64-be72-8bb7e6cc2a60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19f9bf95e41f439c8956f250859575f0', 'request_id': 'req-5737de09-3547-4d64-be72-8bb7e6cc2a60'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:10.731497 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:10.741016 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5737de09-3547-4d64-be72-8bb7e6cc2a60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19f9bf95e41f439c8956f250859575f0', 'request_id': 'req-5737de09-3547-4d64-be72-8bb7e6cc2a60'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:10.821428 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5737de09-3547-4d64-be72-8bb7e6cc2a60 admin admin] The token cache is being invalidate because project 19f9bf95e41f439c8956f250859575f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:25:10.822009 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5737de09-3547-4d64-be72-8bb7e6cc2a60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:10.827826 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5737de09-3547-4d64-be72-8bb7e6cc2a60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:10.845645 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 889/1774] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:25:09 2022] DELETE /identity/v3/projects/19f9bf95e41f439c8956f250859575f0 => generated 0 bytes in 940 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:10.847482 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5737de09-3547-4d64-be72-8bb7e6cc2a60 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:10.857562 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5737de09-3547-4d64-be72-8bb7e6cc2a60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:10.943831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe818c-4fc5-4dca-a3c4-c89aae631ccc None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:11.169035 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5737de09-3547-4d64-be72-8bb7e6cc2a60 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:11.192412 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdcda32-f75a-47f0-8f21-c6ef83ab0925 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:11.193473 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcda32-f75a-47f0-8f21-c6ef83ab0925 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:11.193921 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcda32-f75a-47f0-8f21-c6ef83ab0925 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:11.194370 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcda32-f75a-47f0-8f21-c6ef83ab0925 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:11.205625 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5bdcda32-f75a-47f0-8f21-c6ef83ab0925 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:11.283830 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:11.314162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c437274-70eb-46d0-8bb7-fb875611f626 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:11.315208 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c437274-70eb-46d0-8bb7-fb875611f626 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:11.315678 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c437274-70eb-46d0-8bb7-fb875611f626 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:11.316133 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c437274-70eb-46d0-8bb7-fb875611f626 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:11.326025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9c437274-70eb-46d0-8bb7-fb875611f626 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:11.524499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdcda32-f75a-47f0-8f21-c6ef83ab0925 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:11.528395 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdcda32-f75a-47f0-8f21-c6ef83ab0925 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:11.584372 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 890/1775] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:25:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 738 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:25:11.600162 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d540a-d630-4073-b5b7-29cd011442e6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:11.600695 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d540a-d630-4073-b5b7-29cd011442e6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:11.601192 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d540a-d630-4073-b5b7-29cd011442e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:11.617759 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c437274-70eb-46d0-8bb7-fb875611f626 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:11.626585 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c437274-70eb-46d0-8bb7-fb875611f626 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:11.655911 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 886/1776] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:25:10 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 1166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:25:11.666740 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9affb-6925-42ab-9434-3952f9f12e91 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:11.667815 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9affb-6925-42ab-9434-3952f9f12e91 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:11.668998 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9affb-6925-42ab-9434-3952f9f12e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:11.734858 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-505d540a-d630-4073-b5b7-29cd011442e6 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:11.742080 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-505d540a-d630-4073-b5b7-29cd011442e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:11.753256 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8dc9affb-6925-42ab-9434-3952f9f12e91 None None] MFA Rules not processed for user `6ecf34ceee294b53b5be7c1723b42950`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:11.763630 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8dc9affb-6925-42ab-9434-3952f9f12e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:11.792424 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 891/1777] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:25:11.805400 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37adf8ab-0304-4c72-9d24-d3533a60a55e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:11.808845 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37adf8ab-0304-4c72-9d24-d3533a60a55e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:11.809635 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37adf8ab-0304-4c72-9d24-d3533a60a55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:11.941745 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-37adf8ab-0304-4c72-9d24-d3533a60a55e None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:11.944838 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-37adf8ab-0304-4c72-9d24-d3533a60a55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:12.019305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 892/1778] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:25:12.031494 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12034896-0da2-4398-b327-cb34713d838a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:12.040295 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12034896-0da2-4398-b327-cb34713d838a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:12.043032 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12034896-0da2-4398-b327-cb34713d838a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:12.148425 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 887/1779] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 492 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:25:12.149968 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9affb-6925-42ab-9434-3952f9f12e91 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:12.195114 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-12034896-0da2-4398-b327-cb34713d838a None None] MFA Rules not processed for user `0ed3ed6183d34c73ba40d13af037f838`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:12.197914 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-12034896-0da2-4398-b327-cb34713d838a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:12.227927 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9affb-6925-42ab-9434-3952f9f12e91 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:12.267645 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272712d5-0eb0-4256-88ca-8c59de2a09d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:12.267645 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272712d5-0eb0-4256-88ca-8c59de2a09d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:12.267645 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272712d5-0eb0-4256-88ca-8c59de2a09d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:12.267645 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272712d5-0eb0-4256-88ca-8c59de2a09d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:12.268311 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272712d5-0eb0-4256-88ca-8c59de2a09d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:25:12.268311 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272712d5-0eb0-4256-88ca-8c59de2a09d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:12.270490 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272712d5-0eb0-4256-88ca-8c59de2a09d0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:12.319736 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 888/1780] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:25:12.327659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272712d5-0eb0-4256-88ca-8c59de2a09d0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:12.358549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272712d5-0eb0-4256-88ca-8c59de2a09d0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:12.408968 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209f128-bc52-407b-8ec8-24da5df997dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:12.410538 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209f128-bc52-407b-8ec8-24da5df997dc admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:12.414377 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209f128-bc52-407b-8ec8-24da5df997dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:12.426393 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209f128-bc52-407b-8ec8-24da5df997dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:12.433157 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c209f128-bc52-407b-8ec8-24da5df997dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:12.441216 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c209f128-bc52-407b-8ec8-24da5df997dc admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:12.668846 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 893/1781] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 648 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:25:12.674456 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 889/1782] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:25:12 2022] POST /identity/v3/projects => generated 387 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:25:12.686613 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12034896-0da2-4398-b327-cb34713d838a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:12.731107 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12034896-0da2-4398-b327-cb34713d838a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:12.919198 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:13.099352 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee91c32-fefb-48ee-8d5e-18d36b4efefc None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:13.103805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee91c32-fefb-48ee-8d5e-18d36b4efefc None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:13.104524 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee91c32-fefb-48ee-8d5e-18d36b4efefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:13.255997 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-9ee91c32-fefb-48ee-8d5e-18d36b4efefc None None] MFA Rules not processed for user `181c84715c094100b414cea3c4919eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:13.266776 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9ee91c32-fefb-48ee-8d5e-18d36b4efefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:13.285127 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e78e0-8894-4b5b-8104-e8abdd5eb515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:13.286566 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e78e0-8894-4b5b-8104-e8abdd5eb515 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:13.287422 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e78e0-8894-4b5b-8104-e8abdd5eb515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:13.288419 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e78e0-8894-4b5b-8104-e8abdd5eb515 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:13.291114 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e78e0-8894-4b5b-8104-e8abdd5eb515 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:13.319781 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e78e0-8894-4b5b-8104-e8abdd5eb515 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:13.405783 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 890/1783] 10.222.0.119 () {58 vars in 1016 bytes} [Mon Jul 25 19:25:13 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:25:13.407106 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee91c32-fefb-48ee-8d5e-18d36b4efefc None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:13.408956 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9ee91c32-fefb-48ee-8d5e-18d36b4efefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:13.418262 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 894/1784] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:25:12 2022] POST /identity/v3/users => generated 294 bytes in 733 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:25:13.422195 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e78e0-8894-4b5b-8104-e8abdd5eb515 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:13.497823 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e78e0-8894-4b5b-8104-e8abdd5eb515 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:13.498875 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:13.534628 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751459b5-7333-4393-8cb3-7d0cbd3ca0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:13.538421 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751459b5-7333-4393-8cb3-7d0cbd3ca0c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:13.539361 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751459b5-7333-4393-8cb3-7d0cbd3ca0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:13.540023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751459b5-7333-4393-8cb3-7d0cbd3ca0c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:13.542095 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751459b5-7333-4393-8cb3-7d0cbd3ca0c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:13.560103 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751459b5-7333-4393-8cb3-7d0cbd3ca0c6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:13.616167 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 895/1785] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:25:13.622531 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751459b5-7333-4393-8cb3-7d0cbd3ca0c6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:13.644563 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee91c32-fefb-48ee-8d5e-18d36b4efefc None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:13.662633 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30eb9b6-5fe9-4dac-8387-065eedcf3a71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:13.663319 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30eb9b6-5fe9-4dac-8387-065eedcf3a71 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:13.663773 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30eb9b6-5fe9-4dac-8387-065eedcf3a71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:13.664061 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30eb9b6-5fe9-4dac-8387-065eedcf3a71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:13.667573 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e30eb9b6-5fe9-4dac-8387-065eedcf3a71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:13.674881 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751459b5-7333-4393-8cb3-7d0cbd3ca0c6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:13.695433 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55664e02-85c4-4f22-a641-e4ed5c100c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:13.703690 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55664e02-85c4-4f22-a641-e4ed5c100c67 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:13.704345 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55664e02-85c4-4f22-a641-e4ed5c100c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:13.704852 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55664e02-85c4-4f22-a641-e4ed5c100c67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:13.706334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55664e02-85c4-4f22-a641-e4ed5c100c67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:13.716576 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55664e02-85c4-4f22-a641-e4ed5c100c67 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:13.811017 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 896/1786] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:25:13.818937 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55664e02-85c4-4f22-a641-e4ed5c100c67 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:13.865062 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55664e02-85c4-4f22-a641-e4ed5c100c67 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:13.936034 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65955a2-1243-43f1-8c42-0bca6cc0c3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:13.941018 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65955a2-1243-43f1-8c42-0bca6cc0c3b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:13.941018 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65955a2-1243-43f1-8c42-0bca6cc0c3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:13.941018 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65955a2-1243-43f1-8c42-0bca6cc0c3b9 admin admin] PATH_INFO: `/v3/projects/a83334f2864c47cdbf7a27ca6eb5cfec/users/c828422507fb48eda0fc686ed840d312/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:14.040209 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65955a2-1243-43f1-8c42-0bca6cc0c3b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a83334f2864c47cdbf7a27ca6eb5cfec, user_id=c828422507fb48eda0fc686ed840d312, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:14.110796 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65955a2-1243-43f1-8c42-0bca6cc0c3b9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:14.197294 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 897/1787] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:25:13 2022] PUT /identity/v3/projects/a83334f2864c47cdbf7a27ca6eb5cfec/users/c828422507fb48eda0fc686ed840d312/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:14.199365 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65955a2-1243-43f1-8c42-0bca6cc0c3b9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:14.206398 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30eb9b6-5fe9-4dac-8387-065eedcf3a71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:14.211073 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b65955a2-1243-43f1-8c42-0bca6cc0c3b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:14.222859 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30eb9b6-5fe9-4dac-8387-065eedcf3a71 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:14.320174 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 891/1788] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:25:13 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 914 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:25:14.335106 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30eb9b6-5fe9-4dac-8387-065eedcf3a71 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:14.345069 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e30eb9b6-5fe9-4dac-8387-065eedcf3a71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:14.377484 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30eb9b6-5fe9-4dac-8387-065eedcf3a71 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:14.574614 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65955a2-1243-43f1-8c42-0bca6cc0c3b9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:14.622900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6160c193-82bb-4278-8fae-044ea264b364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:14.628734 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6160c193-82bb-4278-8fae-044ea264b364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:14.635623 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6160c193-82bb-4278-8fae-044ea264b364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:14.639871 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6160c193-82bb-4278-8fae-044ea264b364 admin admin] PATH_INFO: `/v3/projects/e58f0222b5494864b39bb2eeedcaf28a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:14.646210 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6160c193-82bb-4278-8fae-044ea264b364 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e58f0222b5494864b39bb2eeedcaf28a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:14.663189 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6160c193-82bb-4278-8fae-044ea264b364 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:14.680013 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aab5cd-46e7-4960-8ed7-2da1934fdda6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:14.681548 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aab5cd-46e7-4960-8ed7-2da1934fdda6 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:14.682233 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aab5cd-46e7-4960-8ed7-2da1934fdda6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:14.686329 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aab5cd-46e7-4960-8ed7-2da1934fdda6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:14.731456 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aab5cd-46e7-4960-8ed7-2da1934fdda6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:14.733093 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aab5cd-46e7-4960-8ed7-2da1934fdda6 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:14.795681 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 892/1789] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:25:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:25:14.802967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c81a4-8e38-43cd-8508-c24a89ddd7e6 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:14.803330 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c81a4-8e38-43cd-8508-c24a89ddd7e6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:14.804076 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c81a4-8e38-43cd-8508-c24a89ddd7e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:14.935215 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c89c81a4-8e38-43cd-8508-c24a89ddd7e6 None None] MFA Rules not processed for user `c828422507fb48eda0fc686ed840d312`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:14.941121 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c89c81a4-8e38-43cd-8508-c24a89ddd7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:15.012296 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6160c193-82bb-4278-8fae-044ea264b364 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e58f0222b5494864b39bb2eeedcaf28a', 'request_id': 'req-6160c193-82bb-4278-8fae-044ea264b364'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:15.051388 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6160c193-82bb-4278-8fae-044ea264b364 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e58f0222b5494864b39bb2eeedcaf28a', 'request_id': 'req-6160c193-82bb-4278-8fae-044ea264b364'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:15.095611 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6160c193-82bb-4278-8fae-044ea264b364 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e58f0222b5494864b39bb2eeedcaf28a', 'request_id': 'req-6160c193-82bb-4278-8fae-044ea264b364'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:15.178630 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6160c193-82bb-4278-8fae-044ea264b364 admin admin] The token cache is being invalidate because project e58f0222b5494864b39bb2eeedcaf28a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:25:15.181047 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6160c193-82bb-4278-8fae-044ea264b364 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:15.186247 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6160c193-82bb-4278-8fae-044ea264b364 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:15.192293 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 898/1790] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:25:14 2022] DELETE /identity/v3/projects/e58f0222b5494864b39bb2eeedcaf28a => generated 0 bytes in 993 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:15.196602 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63bdfc-fa02-4c3f-9b5e-84138df623ad None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:15.200160 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63bdfc-fa02-4c3f-9b5e-84138df623ad None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:15.204430 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63bdfc-fa02-4c3f-9b5e-84138df623ad None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:15.215466 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 899/1791] 10.222.0.18 () {54 vars in 912 bytes} [Mon Jul 25 19:25:15 2022] GET /identity => generated 270 bytes in 22 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:25:15.234295 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e63bdfc-fa02-4c3f-9b5e-84138df623ad None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:15.246822 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2e63bdfc-fa02-4c3f-9b5e-84138df623ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:15.269826 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:15.614768 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e63bdfc-fa02-4c3f-9b5e-84138df623ad None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:15.626933 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 893/1792] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 830 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:25:15.629864 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d9c0e-0d67-4279-97ed-6ca0c549ff50 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:15.646162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d9c0e-0d67-4279-97ed-6ca0c549ff50 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:15.659023 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d9c0e-0d67-4279-97ed-6ca0c549ff50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:15.664301 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17306040-ee72-4b33-a05b-9ad1f093a556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:15.675550 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17306040-ee72-4b33-a05b-9ad1f093a556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:15.682429 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17306040-ee72-4b33-a05b-9ad1f093a556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:15.683109 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17306040-ee72-4b33-a05b-9ad1f093a556 admin admin] PATH_INFO: `/v3/users/4e2d1bc2fe564267a0675bcfb5ffb7ad` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:15.701730 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17306040-ee72-4b33-a05b-9ad1f093a556 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e2d1bc2fe564267a0675bcfb5ffb7ad)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:15.721616 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:15.737736 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17306040-ee72-4b33-a05b-9ad1f093a556 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:15.809467 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-391d9c0e-0d67-4279-97ed-6ca0c549ff50 None None] MFA Rules not processed for user `181c84715c094100b414cea3c4919eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:15.821539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-391d9c0e-0d67-4279-97ed-6ca0c549ff50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:16.212465 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-17306040-ee72-4b33-a05b-9ad1f093a556 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e2d1bc2fe564267a0675bcfb5ffb7ad'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:16.288605 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-17306040-ee72-4b33-a05b-9ad1f093a556 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e2d1bc2fe564267a0675bcfb5ffb7ad'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:16.324191 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-17306040-ee72-4b33-a05b-9ad1f093a556 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e2d1bc2fe564267a0675bcfb5ffb7ad'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:16.325570 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-17306040-ee72-4b33-a05b-9ad1f093a556 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e2d1bc2fe564267a0675bcfb5ffb7ad'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:16.327173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-17306040-ee72-4b33-a05b-9ad1f093a556 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e2d1bc2fe564267a0675bcfb5ffb7ad'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:16.424358 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 894/1793] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Jul 25 19:25:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 796 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:25:16.426187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d9c0e-0d67-4279-97ed-6ca0c549ff50 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:16.428748 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 900/1794] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:25:15 2022] DELETE /identity/v3/users/4e2d1bc2fe564267a0675bcfb5ffb7ad => generated 0 bytes in 1198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:16.447855 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-391d9c0e-0d67-4279-97ed-6ca0c549ff50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:16.581275 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d9c0e-0d67-4279-97ed-6ca0c549ff50 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:16.877843 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17306040-ee72-4b33-a05b-9ad1f093a556 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:16.888678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-17306040-ee72-4b33-a05b-9ad1f093a556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:16.974704 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d962ea-2204-482e-aaa8-88a1cb507316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:16.975812 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d962ea-2204-482e-aaa8-88a1cb507316 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:16.976826 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d962ea-2204-482e-aaa8-88a1cb507316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:16.977000 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d962ea-2204-482e-aaa8-88a1cb507316 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:16.977835 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d962ea-2204-482e-aaa8-88a1cb507316 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:16.992062 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d962ea-2204-482e-aaa8-88a1cb507316 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:17.142224 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 895/1795] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:25:16 2022] POST /identity/v3/projects => generated 389 bytes in 712 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:25:17.152082 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d962ea-2204-482e-aaa8-88a1cb507316 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:17.242259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17306040-ee72-4b33-a05b-9ad1f093a556 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:17.459123 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:17.500374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d962ea-2204-482e-aaa8-88a1cb507316 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:17.521222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c9878f-d6da-47d9-9f18-b058c187cc76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:17.523614 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c9878f-d6da-47d9-9f18-b058c187cc76 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:17.524533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c9878f-d6da-47d9-9f18-b058c187cc76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:17.525428 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c9878f-d6da-47d9-9f18-b058c187cc76 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:17.527184 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c9878f-d6da-47d9-9f18-b058c187cc76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:17.533816 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f674377-0581-4d0a-b5bc-ca6aad4bf7ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:17.535146 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f674377-0581-4d0a-b5bc-ca6aad4bf7ed service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:17.535666 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f674377-0581-4d0a-b5bc-ca6aad4bf7ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:17.536187 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f674377-0581-4d0a-b5bc-ca6aad4bf7ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:17.538724 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c9878f-d6da-47d9-9f18-b058c187cc76 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:17.587396 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f674377-0581-4d0a-b5bc-ca6aad4bf7ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:17.593158 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f674377-0581-4d0a-b5bc-ca6aad4bf7ed service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:17.620403 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 901/1796] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:25:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 744 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:25:17.647939 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 896/1797] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:25:17 2022] POST /identity/v3/users => generated 301 bytes in 501 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:25:17.654826 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f674377-0581-4d0a-b5bc-ca6aad4bf7ed service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:17.711944 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f674377-0581-4d0a-b5bc-ca6aad4bf7ed service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:17.738858 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d5316c-524a-4d0f-8e24-4c00888679f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:17.739988 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d5316c-524a-4d0f-8e24-4c00888679f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:17.740515 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d5316c-524a-4d0f-8e24-4c00888679f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:17.741017 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d5316c-524a-4d0f-8e24-4c00888679f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:17.752925 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d5316c-524a-4d0f-8e24-4c00888679f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:17.757556 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d5316c-524a-4d0f-8e24-4c00888679f6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:17.808112 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 902/1798] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:25:17.810973 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c9878f-d6da-47d9-9f18-b058c187cc76 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:17.840057 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c9878f-d6da-47d9-9f18-b058c187cc76 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:17.854390 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ccd05-051c-49ec-a35a-114944c020ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:17.858373 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ccd05-051c-49ec-a35a-114944c020ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:17.861822 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ccd05-051c-49ec-a35a-114944c020ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:17.862605 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ccd05-051c-49ec-a35a-114944c020ba admin admin] PATH_INFO: `/v3/projects/e69cfc3b8b4440758a9753fa42874cc5/users/10956d6200d0446a8d6220cd02a18952/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:17.942915 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ccd05-051c-49ec-a35a-114944c020ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=e69cfc3b8b4440758a9753fa42874cc5, user_id=10956d6200d0446a8d6220cd02a18952, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:17.957353 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ccd05-051c-49ec-a35a-114944c020ba admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:18.002481 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 897/1799] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:25:17 2022] PUT /identity/v3/projects/e69cfc3b8b4440758a9753fa42874cc5/users/10956d6200d0446a8d6220cd02a18952/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:25:18.003715 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d5316c-524a-4d0f-8e24-4c00888679f6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:18.028699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d5316c-524a-4d0f-8e24-4c00888679f6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:18.204200 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed3284d-ddfd-4449-aa8a-5fe1a047a387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:18.205637 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3284d-ddfd-4449-aa8a-5fe1a047a387 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:18.206311 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3284d-ddfd-4449-aa8a-5fe1a047a387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:18.207285 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3284d-ddfd-4449-aa8a-5fe1a047a387 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:18.208915 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed3284d-ddfd-4449-aa8a-5fe1a047a387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:18.215428 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed3284d-ddfd-4449-aa8a-5fe1a047a387 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:18.255034 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 903/1800] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:25:18.267691 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ccd05-051c-49ec-a35a-114944c020ba admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:18.287186 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:18.332845 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ccd05-051c-49ec-a35a-114944c020ba admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:18.363574 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca0517e-c053-4679-8e0a-e6bb6b7df15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:18.367818 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0517e-c053-4679-8e0a-e6bb6b7df15b admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:18.368441 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0517e-c053-4679-8e0a-e6bb6b7df15b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:18.371768 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0517e-c053-4679-8e0a-e6bb6b7df15b admin admin] PATH_INFO: `/v3/projects/e69cfc3b8b4440758a9753fa42874cc5/users/10956d6200d0446a8d6220cd02a18952/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:18.383171 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca0517e-c053-4679-8e0a-e6bb6b7df15b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e69cfc3b8b4440758a9753fa42874cc5, user_id=10956d6200d0446a8d6220cd02a18952, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:18.407136 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca0517e-c053-4679-8e0a-e6bb6b7df15b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:18.439777 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 898/1801] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:25:18 2022] PUT /identity/v3/projects/e69cfc3b8b4440758a9753fa42874cc5/users/10956d6200d0446a8d6220cd02a18952/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:18.466188 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbdc67-8439-476c-9feb-ded40bf9fdd3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:18.466907 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbdc67-8439-476c-9feb-ded40bf9fdd3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:18.467564 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbdc67-8439-476c-9feb-ded40bf9fdd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:18.566731 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-aafbdc67-8439-476c-9feb-ded40bf9fdd3 None None] MFA Rules not processed for user `10956d6200d0446a8d6220cd02a18952`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:18.570571 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-aafbdc67-8439-476c-9feb-ded40bf9fdd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:19.081856 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 904/1802] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:18 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 618 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:25:19.093779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca0517e-c053-4679-8e0a-e6bb6b7df15b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:19.096185 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5ca0517e-c053-4679-8e0a-e6bb6b7df15b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:19.168590 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbdc67-8439-476c-9feb-ded40bf9fdd3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:19.171497 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-aafbdc67-8439-476c-9feb-ded40bf9fdd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:19.392035 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca0517e-c053-4679-8e0a-e6bb6b7df15b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:19.405243 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c73190e-d1b1-416e-b3d0-17fa3aaba9da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:19.406741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73190e-d1b1-416e-b3d0-17fa3aaba9da service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:19.407490 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73190e-d1b1-416e-b3d0-17fa3aaba9da service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:19.409657 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73190e-d1b1-416e-b3d0-17fa3aaba9da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:19.453411 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c73190e-d1b1-416e-b3d0-17fa3aaba9da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:19.455417 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c73190e-d1b1-416e-b3d0-17fa3aaba9da service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:19.484791 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 899/1803] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:25:19 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:25:19.523724 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:19.687391 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbdc67-8439-476c-9feb-ded40bf9fdd3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:19.723708 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0146896b-2e68-42f7-80f3-cf4309d439f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:19.725332 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0146896b-2e68-42f7-80f3-cf4309d439f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:19.726286 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0146896b-2e68-42f7-80f3-cf4309d439f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:19.726479 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0146896b-2e68-42f7-80f3-cf4309d439f5 admin admin] PATH_INFO: `/v3/projects/f4bcb443537541a4b35b237a3f206c05` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:19.734427 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0146896b-2e68-42f7-80f3-cf4309d439f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4bcb443537541a4b35b237a3f206c05)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:19.745022 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0146896b-2e68-42f7-80f3-cf4309d439f5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:19.915809 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0146896b-2e68-42f7-80f3-cf4309d439f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4bcb443537541a4b35b237a3f206c05', 'request_id': 'req-0146896b-2e68-42f7-80f3-cf4309d439f5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:19.936368 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0146896b-2e68-42f7-80f3-cf4309d439f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4bcb443537541a4b35b237a3f206c05', 'request_id': 'req-0146896b-2e68-42f7-80f3-cf4309d439f5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:19.955353 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0146896b-2e68-42f7-80f3-cf4309d439f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4bcb443537541a4b35b237a3f206c05', 'request_id': 'req-0146896b-2e68-42f7-80f3-cf4309d439f5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:19.970084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b9c64-9607-46c3-acad-6d1b1fbd0457 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:19.970573 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b9c64-9607-46c3-acad-6d1b1fbd0457 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:19.971121 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b9c64-9607-46c3-acad-6d1b1fbd0457 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:19.973802 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 900/1804] 10.222.0.18 () {54 vars in 921 bytes} [Mon Jul 25 19:25:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 19:25:20.021112 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0146896b-2e68-42f7-80f3-cf4309d439f5 admin admin] The token cache is being invalidate because project f4bcb443537541a4b35b237a3f206c05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:25:20.040137 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d31f3-2603-4956-af14-44cc3a1ec8b3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:20.040664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d31f3-2603-4956-af14-44cc3a1ec8b3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:20.041762 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d31f3-2603-4956-af14-44cc3a1ec8b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:20.042223 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0146896b-2e68-42f7-80f3-cf4309d439f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:20.046334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0146896b-2e68-42f7-80f3-cf4309d439f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:20.052996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 905/1805] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:25:19 2022] DELETE /identity/v3/projects/f4bcb443537541a4b35b237a3f206c05 => generated 0 bytes in 885 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:20.063411 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6366b-cb53-47f0-910b-c19cfe5ebd44 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:20.064578 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6366b-cb53-47f0-910b-c19cfe5ebd44 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:20.066614 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6366b-cb53-47f0-910b-c19cfe5ebd44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:20.177820 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-16c6366b-cb53-47f0-910b-c19cfe5ebd44 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:20.190210 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-16c6366b-cb53-47f0-910b-c19cfe5ebd44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:20.219473 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-9a4d31f3-2603-4956-af14-44cc3a1ec8b3 None None] MFA Rules not processed for user `860e7d26ad364795bb25836aadd4ac7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:20.242905 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9a4d31f3-2603-4956-af14-44cc3a1ec8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:20.625948 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 906/1806] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 566 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:25:20.627915 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6366b-cb53-47f0-910b-c19cfe5ebd44 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:20.639542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-16c6366b-cb53-47f0-910b-c19cfe5ebd44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:20.748487 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6366b-cb53-47f0-910b-c19cfe5ebd44 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:20.799874 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d8e77a-55e8-49ec-a70d-8dee07936c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:20.806129 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8e77a-55e8-49ec-a70d-8dee07936c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:20.807331 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8e77a-55e8-49ec-a70d-8dee07936c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:20.808104 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8e77a-55e8-49ec-a70d-8dee07936c82 admin admin] PATH_INFO: `/v3/users/c828422507fb48eda0fc686ed840d312` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:20.813001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d8e77a-55e8-49ec-a70d-8dee07936c82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c828422507fb48eda0fc686ed840d312)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:20.816688 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d8e77a-55e8-49ec-a70d-8dee07936c82 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:20.880514 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 901/1807] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Jul 25 19:25:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 842 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:25:20.890114 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ef127-2b40-4094-8c95-7692ae4fafd3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:20.890948 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ef127-2b40-4094-8c95-7692ae4fafd3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:20.891711 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ef127-2b40-4094-8c95-7692ae4fafd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:21.057932 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-586ef127-2b40-4094-8c95-7692ae4fafd3 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:21.060169 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-586ef127-2b40-4094-8c95-7692ae4fafd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:21.077354 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d1d8e77a-55e8-49ec-a70d-8dee07936c82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c828422507fb48eda0fc686ed840d312'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:21.159002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 902/1808] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:25:21.162410 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d1d8e77a-55e8-49ec-a70d-8dee07936c82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c828422507fb48eda0fc686ed840d312'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:21.170938 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ef127-2b40-4094-8c95-7692ae4fafd3 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:21.200008 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d1d8e77a-55e8-49ec-a70d-8dee07936c82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c828422507fb48eda0fc686ed840d312'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:21.201779 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d1d8e77a-55e8-49ec-a70d-8dee07936c82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c828422507fb48eda0fc686ed840d312'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:21.205230 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d1d8e77a-55e8-49ec-a70d-8dee07936c82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c828422507fb48eda0fc686ed840d312'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:21.248083 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ef127-2b40-4094-8c95-7692ae4fafd3 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:21.279111 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:21.288803 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64575694-f779-4fa1-a964-a5fa28509a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:21.289527 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64575694-f779-4fa1-a964-a5fa28509a1f admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:21.289821 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64575694-f779-4fa1-a964-a5fa28509a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:21.290110 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64575694-f779-4fa1-a964-a5fa28509a1f admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:21.304967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64575694-f779-4fa1-a964-a5fa28509a1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:25:21.305550 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64575694-f779-4fa1-a964-a5fa28509a1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:21.318684 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64575694-f779-4fa1-a964-a5fa28509a1f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:21.320541 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 907/1809] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:25:20 2022] DELETE /identity/v3/users/c828422507fb48eda0fc686ed840d312 => generated 0 bytes in 693 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:21.330685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862d078-6e0d-48c6-9335-9d8f1cb01314 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:21.331351 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862d078-6e0d-48c6-9335-9d8f1cb01314 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:21.332085 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862d078-6e0d-48c6-9335-9d8f1cb01314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:21.414345 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 903/1810] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:25:21.421522 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64575694-f779-4fa1-a964-a5fa28509a1f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:21.430776 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-64575694-f779-4fa1-a964-a5fa28509a1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:21.441776 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3862d078-6e0d-48c6-9335-9d8f1cb01314 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:21.475769 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3862d078-6e0d-48c6-9335-9d8f1cb01314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:21.518272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64575694-f779-4fa1-a964-a5fa28509a1f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:21.751124 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657440c-d35c-4284-b9c7-9028d9239f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:21.752546 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657440c-d35c-4284-b9c7-9028d9239f86 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:21.752858 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657440c-d35c-4284-b9c7-9028d9239f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:21.757788 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657440c-d35c-4284-b9c7-9028d9239f86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:21.764795 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657440c-d35c-4284-b9c7-9028d9239f86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:21.790100 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657440c-d35c-4284-b9c7-9028d9239f86 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:21.865608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:21.960964 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 904/1811] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:25:21 2022] POST /identity/v3/projects => generated 387 bytes in 539 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:25:21.971754 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657440c-d35c-4284-b9c7-9028d9239f86 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:22.086843 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 908/1812] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 759 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:25:22.268950 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862d078-6e0d-48c6-9335-9d8f1cb01314 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:22.271471 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657440c-d35c-4284-b9c7-9028d9239f86 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:22.272336 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3862d078-6e0d-48c6-9335-9d8f1cb01314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:22.294524 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad8702-a850-4874-a5ae-35dcffe9c9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:22.296130 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad8702-a850-4874-a5ae-35dcffe9c9bf admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:22.296905 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad8702-a850-4874-a5ae-35dcffe9c9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:22.297625 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad8702-a850-4874-a5ae-35dcffe9c9bf admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:22.299390 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ad8702-a850-4874-a5ae-35dcffe9c9bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:22.313860 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ad8702-a850-4874-a5ae-35dcffe9c9bf admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:22.399038 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 905/1813] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:25:21 2022] POST /identity/v3/users => generated 294 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:25:22.401461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad8702-a850-4874-a5ae-35dcffe9c9bf admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:22.451541 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad8702-a850-4874-a5ae-35dcffe9c9bf admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:22.501287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a19dfa-db1c-4e03-b4fd-66bb928336e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:22.502566 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a19dfa-db1c-4e03-b4fd-66bb928336e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:22.503133 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a19dfa-db1c-4e03-b4fd-66bb928336e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:22.503733 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a19dfa-db1c-4e03-b4fd-66bb928336e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:22.519680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a19dfa-db1c-4e03-b4fd-66bb928336e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:22.522806 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a19dfa-db1c-4e03-b4fd-66bb928336e7 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:22.594352 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 906/1814] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:25:22.602726 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a19dfa-db1c-4e03-b4fd-66bb928336e7 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:22.668184 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862d078-6e0d-48c6-9335-9d8f1cb01314 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:22.680090 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a19dfa-db1c-4e03-b4fd-66bb928336e7 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:22.729005 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073d3c0-07aa-4f64-93bc-0282877a762e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:22.732962 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073d3c0-07aa-4f64-93bc-0282877a762e admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:22.736466 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073d3c0-07aa-4f64-93bc-0282877a762e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:22.742696 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073d3c0-07aa-4f64-93bc-0282877a762e admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:22.744934 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5073d3c0-07aa-4f64-93bc-0282877a762e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:22.756100 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5073d3c0-07aa-4f64-93bc-0282877a762e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:22.852168 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 907/1815] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:25:22.854453 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073d3c0-07aa-4f64-93bc-0282877a762e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:22.920508 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073d3c0-07aa-4f64-93bc-0282877a762e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:22.951274 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948668a-99b2-484b-9a21-ba5b9e1b53da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:22.951274 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948668a-99b2-484b-9a21-ba5b9e1b53da admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:22.951274 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948668a-99b2-484b-9a21-ba5b9e1b53da admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:22.951274 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948668a-99b2-484b-9a21-ba5b9e1b53da admin admin] PATH_INFO: `/v3/projects/e5d6e499a2af484f9946907769101c63/users/14dc9d6a27aa45f9886d6c734165992a/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:23.055497 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa2fe2a-78f7-43e5-b8ce-305e21bc7fc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:23.060093 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2fe2a-78f7-43e5-b8ce-305e21bc7fc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:23.060583 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2fe2a-78f7-43e5-b8ce-305e21bc7fc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:23.063054 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2fe2a-78f7-43e5-b8ce-305e21bc7fc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:23.128818 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa2fe2a-78f7-43e5-b8ce-305e21bc7fc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:23.133420 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa2fe2a-78f7-43e5-b8ce-305e21bc7fc6 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:23.201335 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d948668a-99b2-484b-9a21-ba5b9e1b53da admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5d6e499a2af484f9946907769101c63, user_id=14dc9d6a27aa45f9886d6c734165992a, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:23.206358 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d948668a-99b2-484b-9a21-ba5b9e1b53da admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:23.208738 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 909/1816] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:25:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 942 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:25:23.209609 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa2fe2a-78f7-43e5-b8ce-305e21bc7fc6 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:23.231446 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ffa2fe2a-78f7-43e5-b8ce-305e21bc7fc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:23.296314 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 908/1817] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:25:22 2022] PUT /identity/v3/projects/e5d6e499a2af484f9946907769101c63/users/14dc9d6a27aa45f9886d6c734165992a/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:23.334653 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f010a-a849-4b34-827c-dcd669922e80 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:23.335108 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f010a-a849-4b34-827c-dcd669922e80 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:23.335904 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f010a-a849-4b34-827c-dcd669922e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:23.473564 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e03f010a-a849-4b34-827c-dcd669922e80 None None] MFA Rules not processed for user `14dc9d6a27aa45f9886d6c734165992a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:23.487343 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e03f010a-a849-4b34-827c-dcd669922e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:23.755958 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:23.888190 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 909/1818] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 554 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:25:23.897247 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f010a-a849-4b34-827c-dcd669922e80 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:23.932144 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa2fe2a-78f7-43e5-b8ce-305e21bc7fc6 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:24.285123 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:24.353676 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dd8417-6568-40c2-b7b3-dc0401b44cee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:24.356290 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd8417-6568-40c2-b7b3-dc0401b44cee service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:24.357473 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd8417-6568-40c2-b7b3-dc0401b44cee service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:24.358364 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd8417-6568-40c2-b7b3-dc0401b44cee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:24.364484 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-39dd8417-6568-40c2-b7b3-dc0401b44cee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:24.482521 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dd8417-6568-40c2-b7b3-dc0401b44cee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:24.514714 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dd8417-6568-40c2-b7b3-dc0401b44cee service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:24.585398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 910/1819] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:25:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 1376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:25:24.611013 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f010a-a849-4b34-827c-dcd669922e80 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:24.665434 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb4eaa-95e0-409f-8664-06cf7dc3a730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:24.668391 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb4eaa-95e0-409f-8664-06cf7dc3a730 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:24.669025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb4eaa-95e0-409f-8664-06cf7dc3a730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:24.669599 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb4eaa-95e0-409f-8664-06cf7dc3a730 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:24.701215 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb4eaa-95e0-409f-8664-06cf7dc3a730 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:24.711702 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb4eaa-95e0-409f-8664-06cf7dc3a730 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:24.932597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 910/1820] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:25:23 2022] POST /identity/v3/projects => generated 387 bytes in 1043 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:25:24.939744 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dd8417-6568-40c2-b7b3-dc0401b44cee service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:24.990960 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dd8417-6568-40c2-b7b3-dc0401b44cee service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:25.332448 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dcf5f5-8404-45a2-b7f8-65e7c389f978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:25.337340 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dcf5f5-8404-45a2-b7f8-65e7c389f978 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:25.339593 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dcf5f5-8404-45a2-b7f8-65e7c389f978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:25.340470 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dcf5f5-8404-45a2-b7f8-65e7c389f978 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:25.343315 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dcf5f5-8404-45a2-b7f8-65e7c389f978 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:25.350056 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dcf5f5-8404-45a2-b7f8-65e7c389f978 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:25.439960 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 911/1821] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:25:24 2022] POST /identity/v3/users => generated 300 bytes in 501 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:25:25.446363 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb4eaa-95e0-409f-8664-06cf7dc3a730 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:25.514271 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb4eaa-95e0-409f-8664-06cf7dc3a730 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:25.585126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3c8632-d4d2-4ce5-8ea2-cc3affc93c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:25.592754 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c8632-d4d2-4ce5-8ea2-cc3affc93c30 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:25.597186 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c8632-d4d2-4ce5-8ea2-cc3affc93c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:25.597848 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c8632-d4d2-4ce5-8ea2-cc3affc93c30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:25.602665 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3c8632-d4d2-4ce5-8ea2-cc3affc93c30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:25.611560 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3c8632-d4d2-4ce5-8ea2-cc3affc93c30 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:25.657258 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 911/1822] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:25:25.670275 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dcf5f5-8404-45a2-b7f8-65e7c389f978 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:25.749347 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dcf5f5-8404-45a2-b7f8-65e7c389f978 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:25.765563 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2982b-05da-48dd-9f3e-fdd93d2f4837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:25.767841 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2982b-05da-48dd-9f3e-fdd93d2f4837 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:25.769270 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2982b-05da-48dd-9f3e-fdd93d2f4837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:25.770096 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2982b-05da-48dd-9f3e-fdd93d2f4837 admin admin] PATH_INFO: `/v3/projects/d763543db78a4a7a9304fd3652704880/users/db263232b13646ddbb061aa4d6236522/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:25.828743 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e2982b-05da-48dd-9f3e-fdd93d2f4837 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d763543db78a4a7a9304fd3652704880, user_id=db263232b13646ddbb061aa4d6236522, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:25.843789 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e2982b-05da-48dd-9f3e-fdd93d2f4837 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:25.894805 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 912/1823] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:25:25 2022] PUT /identity/v3/projects/d763543db78a4a7a9304fd3652704880/users/db263232b13646ddbb061aa4d6236522/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:25.896641 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3c8632-d4d2-4ce5-8ea2-cc3affc93c30 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:25.942115 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3c8632-d4d2-4ce5-8ea2-cc3affc93c30 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:26.168266 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2addb38-ccef-419b-99dc-5436077a3a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:26.169557 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2addb38-ccef-419b-99dc-5436077a3a2d admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:26.170131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2addb38-ccef-419b-99dc-5436077a3a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:26.170595 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2addb38-ccef-419b-99dc-5436077a3a2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:26.171629 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2addb38-ccef-419b-99dc-5436077a3a2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:26.174849 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2addb38-ccef-419b-99dc-5436077a3a2d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:26.208464 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2982b-05da-48dd-9f3e-fdd93d2f4837 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:26.220452 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 912/1824] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:25:26.230885 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2982b-05da-48dd-9f3e-fdd93d2f4837 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:26.245725 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd01005b-8591-4710-9669-7f2feb189cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:26.246747 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01005b-8591-4710-9669-7f2feb189cb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:26.248403 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01005b-8591-4710-9669-7f2feb189cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:26.250283 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01005b-8591-4710-9669-7f2feb189cb2 admin admin] PATH_INFO: `/v3/projects/d763543db78a4a7a9304fd3652704880/users/db263232b13646ddbb061aa4d6236522/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:26.276893 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd01005b-8591-4710-9669-7f2feb189cb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d763543db78a4a7a9304fd3652704880, user_id=db263232b13646ddbb061aa4d6236522, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:26.281777 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd01005b-8591-4710-9669-7f2feb189cb2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:26.323271 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 913/1825] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:25:26 2022] PUT /identity/v3/projects/d763543db78a4a7a9304fd3652704880/users/db263232b13646ddbb061aa4d6236522/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:26.355809 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d257430-3839-428a-8fd2-79de2c1b91cf None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:26.355809 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d257430-3839-428a-8fd2-79de2c1b91cf None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:26.355809 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d257430-3839-428a-8fd2-79de2c1b91cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:26.389880 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd01005b-8591-4710-9669-7f2feb189cb2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:26.392968 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bd01005b-8591-4710-9669-7f2feb189cb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:26.517135 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0d257430-3839-428a-8fd2-79de2c1b91cf None None] MFA Rules not processed for user `db263232b13646ddbb061aa4d6236522`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:26.519376 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0d257430-3839-428a-8fd2-79de2c1b91cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:26.553368 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:26.721510 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd01005b-8591-4710-9669-7f2feb189cb2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:26.741043 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c996abe3-0432-45b0-ae4d-cf02fc7ecad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:26.742026 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c996abe3-0432-45b0-ae4d-cf02fc7ecad4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:26.742336 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c996abe3-0432-45b0-ae4d-cf02fc7ecad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:26.742607 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c996abe3-0432-45b0-ae4d-cf02fc7ecad4 admin admin] PATH_INFO: `/v3/projects/a83334f2864c47cdbf7a27ca6eb5cfec` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:26.746719 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c996abe3-0432-45b0-ae4d-cf02fc7ecad4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a83334f2864c47cdbf7a27ca6eb5cfec)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:26.750446 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c996abe3-0432-45b0-ae4d-cf02fc7ecad4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:26.846978 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 913/1826] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:26 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 502 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:25:26.874045 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d257430-3839-428a-8fd2-79de2c1b91cf None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:26.876578 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0d257430-3839-428a-8fd2-79de2c1b91cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:26.924965 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c996abe3-0432-45b0-ae4d-cf02fc7ecad4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a83334f2864c47cdbf7a27ca6eb5cfec', 'request_id': 'req-c996abe3-0432-45b0-ae4d-cf02fc7ecad4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:26.932265 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:26.945602 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c996abe3-0432-45b0-ae4d-cf02fc7ecad4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a83334f2864c47cdbf7a27ca6eb5cfec', 'request_id': 'req-c996abe3-0432-45b0-ae4d-cf02fc7ecad4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:26.972909 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c996abe3-0432-45b0-ae4d-cf02fc7ecad4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a83334f2864c47cdbf7a27ca6eb5cfec', 'request_id': 'req-c996abe3-0432-45b0-ae4d-cf02fc7ecad4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:27.017495 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c996abe3-0432-45b0-ae4d-cf02fc7ecad4 admin admin] The token cache is being invalidate because project a83334f2864c47cdbf7a27ca6eb5cfec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:25:27.017857 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c996abe3-0432-45b0-ae4d-cf02fc7ecad4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:27.018710 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c996abe3-0432-45b0-ae4d-cf02fc7ecad4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:27.023558 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 914/1827] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:25:26 2022] DELETE /identity/v3/projects/a83334f2864c47cdbf7a27ca6eb5cfec => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:25:27.028420 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c996abe3-0432-45b0-ae4d-cf02fc7ecad4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:27.031677 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c996abe3-0432-45b0-ae4d-cf02fc7ecad4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:27.045973 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d257430-3839-428a-8fd2-79de2c1b91cf None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:27.208683 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d1bf6-cb0b-4428-981a-b18e39fea01b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:27.210523 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d1bf6-cb0b-4428-981a-b18e39fea01b service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:27.210940 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d1bf6-cb0b-4428-981a-b18e39fea01b service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:27.212007 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d1bf6-cb0b-4428-981a-b18e39fea01b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:27.219472 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-287d1bf6-cb0b-4428-981a-b18e39fea01b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:27.321361 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c996abe3-0432-45b0-ae4d-cf02fc7ecad4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:27.331267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14483480-d205-4153-8971-8e2c90e6d369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:27.332340 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14483480-d205-4153-8971-8e2c90e6d369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:27.332704 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14483480-d205-4153-8971-8e2c90e6d369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:27.334107 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14483480-d205-4153-8971-8e2c90e6d369 admin admin] PATH_INFO: `/v3/users/10956d6200d0446a8d6220cd02a18952` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:27.334971 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14483480-d205-4153-8971-8e2c90e6d369 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10956d6200d0446a8d6220cd02a18952)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:27.337108 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14483480-d205-4153-8971-8e2c90e6d369 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:27.397180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d1bf6-cb0b-4428-981a-b18e39fea01b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:27.398311 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d1bf6-cb0b-4428-981a-b18e39fea01b service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:27.439877 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 914/1828] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:25:26 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:25:27.487897 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-14483480-d205-4153-8971-8e2c90e6d369 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10956d6200d0446a8d6220cd02a18952'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:27.511924 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-14483480-d205-4153-8971-8e2c90e6d369 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10956d6200d0446a8d6220cd02a18952'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:27.521395 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:27.536589 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-14483480-d205-4153-8971-8e2c90e6d369 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10956d6200d0446a8d6220cd02a18952'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:27.538426 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-14483480-d205-4153-8971-8e2c90e6d369 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10956d6200d0446a8d6220cd02a18952'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:27.539711 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-14483480-d205-4153-8971-8e2c90e6d369 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10956d6200d0446a8d6220cd02a18952'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:27.586306 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 915/1829] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:25:27 2022] DELETE /identity/v3/users/10956d6200d0446a8d6220cd02a18952 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:27.612565 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d1bf6-cb0b-4428-981a-b18e39fea01b service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:27.616579 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-287d1bf6-cb0b-4428-981a-b18e39fea01b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:27.655969 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d1bf6-cb0b-4428-981a-b18e39fea01b service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:27.831999 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603b4836-d627-42bd-98e9-94643da97dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:27.833549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b4836-d627-42bd-98e9-94643da97dc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:27.834025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b4836-d627-42bd-98e9-94643da97dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:27.834512 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b4836-d627-42bd-98e9-94643da97dc5 admin admin] PATH_INFO: `/v3/users/14dc9d6a27aa45f9886d6c734165992a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:27.836971 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603b4836-d627-42bd-98e9-94643da97dc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14dc9d6a27aa45f9886d6c734165992a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:27.841710 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603b4836-d627-42bd-98e9-94643da97dc5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:27.915095 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:27.956410 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-603b4836-d627-42bd-98e9-94643da97dc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14dc9d6a27aa45f9886d6c734165992a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:27.975278 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-603b4836-d627-42bd-98e9-94643da97dc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14dc9d6a27aa45f9886d6c734165992a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:28.000829 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-603b4836-d627-42bd-98e9-94643da97dc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14dc9d6a27aa45f9886d6c734165992a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:28.003423 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-603b4836-d627-42bd-98e9-94643da97dc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14dc9d6a27aa45f9886d6c734165992a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:28.036117 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-603b4836-d627-42bd-98e9-94643da97dc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14dc9d6a27aa45f9886d6c734165992a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:28.052685 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 915/1830] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:25:27 2022] DELETE /identity/v3/users/14dc9d6a27aa45f9886d6c734165992a => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:28.058981 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca0ba8-b89d-43b5-bc59-63f14054a52c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:28.058981 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca0ba8-b89d-43b5-bc59-63f14054a52c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:28.059660 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca0ba8-b89d-43b5-bc59-63f14054a52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:28.144012 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-65ca0ba8-b89d-43b5-bc59-63f14054a52c None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:28.145634 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-65ca0ba8-b89d-43b5-bc59-63f14054a52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:28.189045 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603b4836-d627-42bd-98e9-94643da97dc5 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:28.192570 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-603b4836-d627-42bd-98e9-94643da97dc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:28.366799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 916/1831] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:25:28.379879 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca0ba8-b89d-43b5-bc59-63f14054a52c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:28.382132 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-65ca0ba8-b89d-43b5-bc59-63f14054a52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:28.502573 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603b4836-d627-42bd-98e9-94643da97dc5 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:28.512571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cc4125-f8f7-4c44-8b26-b7faec7c0079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:28.514069 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc4125-f8f7-4c44-8b26-b7faec7c0079 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:28.514565 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc4125-f8f7-4c44-8b26-b7faec7c0079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:28.515060 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cc4125-f8f7-4c44-8b26-b7faec7c0079 admin admin] PATH_INFO: `/v3/projects/e69cfc3b8b4440758a9753fa42874cc5` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:28.519309 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc4125-f8f7-4c44-8b26-b7faec7c0079 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e69cfc3b8b4440758a9753fa42874cc5)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:28.522969 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cc4125-f8f7-4c44-8b26-b7faec7c0079 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:28.570333 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca0ba8-b89d-43b5-bc59-63f14054a52c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:28.587060 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d7cac-9a2a-482f-9c38-add7def766d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:28.588446 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d7cac-9a2a-482f-9c38-add7def766d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:28.588843 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d7cac-9a2a-482f-9c38-add7def766d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:28.589640 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d7cac-9a2a-482f-9c38-add7def766d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:28.627246 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d7cac-9a2a-482f-9c38-add7def766d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:28.629953 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d7cac-9a2a-482f-9c38-add7def766d5 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:28.655602 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 917/1832] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:25:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:25:28.751370 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-14cc4125-f8f7-4c44-8b26-b7faec7c0079 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e69cfc3b8b4440758a9753fa42874cc5', 'request_id': 'req-14cc4125-f8f7-4c44-8b26-b7faec7c0079'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:28.776350 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-14cc4125-f8f7-4c44-8b26-b7faec7c0079 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e69cfc3b8b4440758a9753fa42874cc5', 'request_id': 'req-14cc4125-f8f7-4c44-8b26-b7faec7c0079'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:28.808925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-14cc4125-f8f7-4c44-8b26-b7faec7c0079 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e69cfc3b8b4440758a9753fa42874cc5', 'request_id': 'req-14cc4125-f8f7-4c44-8b26-b7faec7c0079'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:28.837578 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-14cc4125-f8f7-4c44-8b26-b7faec7c0079 admin admin] The token cache is being invalidate because project e69cfc3b8b4440758a9753fa42874cc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:25:28.839774 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-14cc4125-f8f7-4c44-8b26-b7faec7c0079 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:28.841201 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-14cc4125-f8f7-4c44-8b26-b7faec7c0079 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:28.844768 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 916/1833] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:25:28 2022] DELETE /identity/v3/projects/e69cfc3b8b4440758a9753fa42874cc5 => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:28.854918 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae626a7-c383-4f67-adeb-9fb5f43de642 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:28.855648 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae626a7-c383-4f67-adeb-9fb5f43de642 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:28.856301 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae626a7-c383-4f67-adeb-9fb5f43de642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:28.933896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4ae626a7-c383-4f67-adeb-9fb5f43de642 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:28.939476 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4ae626a7-c383-4f67-adeb-9fb5f43de642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:29.182722 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 918/1834] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:25:29.195430 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b173810-60fc-4d8c-85c7-dddced58a83d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:29.196341 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b173810-60fc-4d8c-85c7-dddced58a83d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:29.196928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b173810-60fc-4d8c-85c7-dddced58a83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:29.326061 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1b173810-60fc-4d8c-85c7-dddced58a83d None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:29.327918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1b173810-60fc-4d8c-85c7-dddced58a83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:29.347169 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 917/1835] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:25:29.350867 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae626a7-c383-4f67-adeb-9fb5f43de642 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:29.366473 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae626a7-c383-4f67-adeb-9fb5f43de642 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:29.374417 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d0023-b482-4b5e-9593-4719a13082e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:29.375444 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d0023-b482-4b5e-9593-4719a13082e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:29.375444 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d0023-b482-4b5e-9593-4719a13082e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:29.375788 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d0023-b482-4b5e-9593-4719a13082e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:29.377142 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d0023-b482-4b5e-9593-4719a13082e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:25:29.378520 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d0023-b482-4b5e-9593-4719a13082e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:29.380505 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d0023-b482-4b5e-9593-4719a13082e6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:29.407122 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 919/1836] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:25:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:25:29.410082 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b173810-60fc-4d8c-85c7-dddced58a83d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:29.423306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b173810-60fc-4d8c-85c7-dddced58a83d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:29.429324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e9b7f-c62e-4896-9cfc-e0afff68f605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:29.430142 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e9b7f-c62e-4896-9cfc-e0afff68f605 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:29.430428 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e9b7f-c62e-4896-9cfc-e0afff68f605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:29.430705 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e9b7f-c62e-4896-9cfc-e0afff68f605 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:29.432234 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719e9b7f-c62e-4896-9cfc-e0afff68f605 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:29.435586 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719e9b7f-c62e-4896-9cfc-e0afff68f605 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:29.489695 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 918/1837] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:25:29 2022] POST /identity/v3/projects => generated 407 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:25:29.495452 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d0023-b482-4b5e-9593-4719a13082e6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:29.508809 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d0023-b482-4b5e-9593-4719a13082e6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:29.517785 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:29.522213 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e9b7f-c62e-4896-9cfc-e0afff68f605 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:29.525264 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-719e9b7f-c62e-4896-9cfc-e0afff68f605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:29.535034 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:29.662452 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef9adc-eeed-42fe-ad48-f906c2e6d643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:29.663158 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef9adc-eeed-42fe-ad48-f906c2e6d643 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:29.663448 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef9adc-eeed-42fe-ad48-f906c2e6d643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:29.663788 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef9adc-eeed-42fe-ad48-f906c2e6d643 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:29.664504 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef9adc-eeed-42fe-ad48-f906c2e6d643 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:29.665203 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e9b7f-c62e-4896-9cfc-e0afff68f605 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:29.667294 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef9adc-eeed-42fe-ad48-f906c2e6d643 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:29.672237 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737d326-a858-47f9-93e3-176374bd1ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:29.673299 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737d326-a858-47f9-93e3-176374bd1ea7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:29.673695 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737d326-a858-47f9-93e3-176374bd1ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:29.674106 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737d326-a858-47f9-93e3-176374bd1ea7 admin admin] PATH_INFO: `/v3/projects/e5d6e499a2af484f9946907769101c63` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:29.675934 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0737d326-a858-47f9-93e3-176374bd1ea7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5d6e499a2af484f9946907769101c63)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:29.678590 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0737d326-a858-47f9-93e3-176374bd1ea7 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:29.702455 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 920/1838] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:25:29 2022] POST /identity/v3/users => generated 304 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:25:29.708512 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef9adc-eeed-42fe-ad48-f906c2e6d643 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:29.727283 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef9adc-eeed-42fe-ad48-f906c2e6d643 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:29.737562 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525cb1d-fead-4136-a8bb-4c42b7cf6651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:29.738544 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525cb1d-fead-4136-a8bb-4c42b7cf6651 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:29.738912 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525cb1d-fead-4136-a8bb-4c42b7cf6651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:29.739456 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525cb1d-fead-4136-a8bb-4c42b7cf6651 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:29.740314 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6525cb1d-fead-4136-a8bb-4c42b7cf6651 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:29.742678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6525cb1d-fead-4136-a8bb-4c42b7cf6651 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:29.757683 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0737d326-a858-47f9-93e3-176374bd1ea7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5d6e499a2af484f9946907769101c63', 'request_id': 'req-0737d326-a858-47f9-93e3-176374bd1ea7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:29.761411 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 921/1839] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:25:29 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:25:29.767244 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525cb1d-fead-4136-a8bb-4c42b7cf6651 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:29.775210 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0737d326-a858-47f9-93e3-176374bd1ea7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5d6e499a2af484f9946907769101c63', 'request_id': 'req-0737d326-a858-47f9-93e3-176374bd1ea7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:29.780550 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525cb1d-fead-4136-a8bb-4c42b7cf6651 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:29.785978 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0737d326-a858-47f9-93e3-176374bd1ea7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5d6e499a2af484f9946907769101c63', 'request_id': 'req-0737d326-a858-47f9-93e3-176374bd1ea7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:29.804493 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0737d326-a858-47f9-93e3-176374bd1ea7 admin admin] The token cache is being invalidate because project e5d6e499a2af484f9946907769101c63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:25:29.804797 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0737d326-a858-47f9-93e3-176374bd1ea7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:29.805469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0737d326-a858-47f9-93e3-176374bd1ea7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:29.808743 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 919/1840] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:25:29 2022] DELETE /identity/v3/projects/e5d6e499a2af484f9946907769101c63 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:25:29.813207 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737d326-a858-47f9-93e3-176374bd1ea7 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:29.814740 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0737d326-a858-47f9-93e3-176374bd1ea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:29.911261 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e56a3c-4bf1-458e-933f-d1cdfd6f9afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:29.912530 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e56a3c-4bf1-458e-933f-d1cdfd6f9afa admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:29.913038 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e56a3c-4bf1-458e-933f-d1cdfd6f9afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:29.913548 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e56a3c-4bf1-458e-933f-d1cdfd6f9afa admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:29.914995 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e56a3c-4bf1-458e-933f-d1cdfd6f9afa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:29.919219 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e56a3c-4bf1-458e-933f-d1cdfd6f9afa admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:29.935425 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737d326-a858-47f9-93e3-176374bd1ea7 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:29.943290 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 922/1841] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:25:29 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:25:29.948463 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e56a3c-4bf1-458e-933f-d1cdfd6f9afa admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:29.950581 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ee428-55be-46ba-b5b2-a41251b51633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:29.951536 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ee428-55be-46ba-b5b2-a41251b51633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:29.952399 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ee428-55be-46ba-b5b2-a41251b51633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:29.952577 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-39e56a3c-4bf1-458e-933f-d1cdfd6f9afa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:29.956027 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ee428-55be-46ba-b5b2-a41251b51633 admin admin] PATH_INFO: `/v3/users/db263232b13646ddbb061aa4d6236522` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:29.957799 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ee428-55be-46ba-b5b2-a41251b51633 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db263232b13646ddbb061aa4d6236522)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:29.960264 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ee428-55be-46ba-b5b2-a41251b51633 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:29.982103 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e56a3c-4bf1-458e-933f-d1cdfd6f9afa admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:29.993431 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f532a-b29b-4afc-8ec4-0c0d12ca867b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:29.994814 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f532a-b29b-4afc-8ec4-0c0d12ca867b admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:29.995176 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f532a-b29b-4afc-8ec4-0c0d12ca867b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:29.995479 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f532a-b29b-4afc-8ec4-0c0d12ca867b admin admin] PATH_INFO: `/v3/projects/fc52b728c9bf46b998500900987f72a3/users/f63354875bca405199ae42ce032743fa/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:30.021602 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847f532a-b29b-4afc-8ec4-0c0d12ca867b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc52b728c9bf46b998500900987f72a3, user_id=f63354875bca405199ae42ce032743fa, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:30.026552 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847f532a-b29b-4afc-8ec4-0c0d12ca867b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:30.050525 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:30.057154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 923/1842] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:25:29 2022] PUT /identity/v3/projects/fc52b728c9bf46b998500900987f72a3/users/f63354875bca405199ae42ce032743fa/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:25:30.076023 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e48ee428-55be-46ba-b5b2-a41251b51633 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db263232b13646ddbb061aa4d6236522'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:30.077760 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ba8ea-b298-451b-aedc-4191a602d633 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:30.078247 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ba8ea-b298-451b-aedc-4191a602d633 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:30.080080 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ba8ea-b298-451b-aedc-4191a602d633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:30.093418 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e48ee428-55be-46ba-b5b2-a41251b51633 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db263232b13646ddbb061aa4d6236522'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:30.108257 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e48ee428-55be-46ba-b5b2-a41251b51633 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db263232b13646ddbb061aa4d6236522'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:30.110040 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e48ee428-55be-46ba-b5b2-a41251b51633 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db263232b13646ddbb061aa4d6236522'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:30.132618 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ae7ba8ea-b298-451b-aedc-4191a602d633 None None] MFA Rules not processed for user `f63354875bca405199ae42ce032743fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:30.135726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ae7ba8ea-b298-451b-aedc-4191a602d633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:30.144950 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e48ee428-55be-46ba-b5b2-a41251b51633 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db263232b13646ddbb061aa4d6236522'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:30.157816 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 920/1843] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:25:29 2022] DELETE /identity/v3/users/db263232b13646ddbb061aa4d6236522 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:25:30.240507 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:30.290230 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 924/1844] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:30 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:25:30.295925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ee428-55be-46ba-b5b2-a41251b51633 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:30.297769 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e48ee428-55be-46ba-b5b2-a41251b51633 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:30.473146 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ee428-55be-46ba-b5b2-a41251b51633 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:30.481965 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a8ffa9-1e17-483f-af3c-5ccf64c9b9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:30.484191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8ffa9-1e17-483f-af3c-5ccf64c9b9e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:30.484705 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8ffa9-1e17-483f-af3c-5ccf64c9b9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:30.485096 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8ffa9-1e17-483f-af3c-5ccf64c9b9e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:30.486086 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a8ffa9-1e17-483f-af3c-5ccf64c9b9e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:30.512085 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a8ffa9-1e17-483f-af3c-5ccf64c9b9e0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:30.575250 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 921/1845] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:25:30 2022] POST /identity/v3/projects => generated 409 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:25:30.578532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ba8ea-b298-451b-aedc-4191a602d633 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:30.729625 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ba8ea-b298-451b-aedc-4191a602d633 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:30.732539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a8ffa9-1e17-483f-af3c-5ccf64c9b9e0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:30.734722 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c6a8ffa9-1e17-483f-af3c-5ccf64c9b9e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:30.826464 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e50fa-7f9b-4dd9-8725-fc730440809b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:30.827499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e50fa-7f9b-4dd9-8725-fc730440809b admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:30.827899 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e50fa-7f9b-4dd9-8725-fc730440809b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:30.828240 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e50fa-7f9b-4dd9-8725-fc730440809b admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:30.829301 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713e50fa-7f9b-4dd9-8725-fc730440809b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:30.832918 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713e50fa-7f9b-4dd9-8725-fc730440809b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:30.859705 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a8ffa9-1e17-483f-af3c-5ccf64c9b9e0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:30.871944 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 925/1846] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:25:30 2022] POST /identity/v3/users => generated 311 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:25:30.872229 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e9860-c699-4689-8f4a-e68892a996fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:30.872229 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e9860-c699-4689-8f4a-e68892a996fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:30.872229 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e9860-c699-4689-8f4a-e68892a996fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:30.872229 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e9860-c699-4689-8f4a-e68892a996fa admin admin] PATH_INFO: `/v3/projects/d763543db78a4a7a9304fd3652704880` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:30.872768 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277e9860-c699-4689-8f4a-e68892a996fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=d763543db78a4a7a9304fd3652704880)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:30.874217 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277e9860-c699-4689-8f4a-e68892a996fa admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:30.874578 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e50fa-7f9b-4dd9-8725-fc730440809b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:30.898159 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e50fa-7f9b-4dd9-8725-fc730440809b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:30.913846 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8ca42-9198-483d-a00a-febcb1284186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:30.914929 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8ca42-9198-483d-a00a-febcb1284186 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:30.915262 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8ca42-9198-483d-a00a-febcb1284186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:30.915621 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8ca42-9198-483d-a00a-febcb1284186 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:30.916445 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f8ca42-9198-483d-a00a-febcb1284186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:30.918826 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f8ca42-9198-483d-a00a-febcb1284186 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:30.933527 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 926/1847] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:25:30.940049 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8ca42-9198-483d-a00a-febcb1284186 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:30.951438 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:30.953317 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8ca42-9198-483d-a00a-febcb1284186 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:30.965115 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-277e9860-c699-4689-8f4a-e68892a996fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd763543db78a4a7a9304fd3652704880', 'request_id': 'req-277e9860-c699-4689-8f4a-e68892a996fa'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:30.979670 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-277e9860-c699-4689-8f4a-e68892a996fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd763543db78a4a7a9304fd3652704880', 'request_id': 'req-277e9860-c699-4689-8f4a-e68892a996fa'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:30.989133 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-277e9860-c699-4689-8f4a-e68892a996fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd763543db78a4a7a9304fd3652704880', 'request_id': 'req-277e9860-c699-4689-8f4a-e68892a996fa'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:31.006826 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-277e9860-c699-4689-8f4a-e68892a996fa admin admin] The token cache is being invalidate because project d763543db78a4a7a9304fd3652704880 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:25:31.007197 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-277e9860-c699-4689-8f4a-e68892a996fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:31.008115 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-277e9860-c699-4689-8f4a-e68892a996fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:31.015630 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 922/1848] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:25:30 2022] DELETE /identity/v3/projects/d763543db78a4a7a9304fd3652704880 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:25:31.019009 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4372f2-7b1d-46d4-80d0-99475029aa0a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:31.019478 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4372f2-7b1d-46d4-80d0-99475029aa0a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:31.020003 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4372f2-7b1d-46d4-80d0-99475029aa0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:31.052899 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ea4372f2-7b1d-46d4-80d0-99475029aa0a None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:31.054196 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ea4372f2-7b1d-46d4-80d0-99475029aa0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:31.054469 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b75a510-b116-450f-9963-7bb2d6f7dc4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:31.055446 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75a510-b116-450f-9963-7bb2d6f7dc4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:31.055758 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75a510-b116-450f-9963-7bb2d6f7dc4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:31.056243 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75a510-b116-450f-9963-7bb2d6f7dc4f admin admin] PATH_INFO: `/v3/projects/37d83134d1c4493a90241f74b348103d/users/7514086938a446cfb360af01aa24b24a/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:31.064189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 923/1849] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:25:31.076118 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee493247-cf4d-4409-a6a1-771c731fc6b6 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:31.076572 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee493247-cf4d-4409-a6a1-771c731fc6b6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:31.077023 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee493247-cf4d-4409-a6a1-771c731fc6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:31.077307 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b75a510-b116-450f-9963-7bb2d6f7dc4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=37d83134d1c4493a90241f74b348103d, user_id=7514086938a446cfb360af01aa24b24a, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:31.080946 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b75a510-b116-450f-9963-7bb2d6f7dc4f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:31.094730 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:31.106050 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 927/1850] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:25:30 2022] PUT /identity/v3/projects/37d83134d1c4493a90241f74b348103d/users/7514086938a446cfb360af01aa24b24a/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:31.111094 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b75a510-b116-450f-9963-7bb2d6f7dc4f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:31.113370 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0b75a510-b116-450f-9963-7bb2d6f7dc4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:31.113688 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ee493247-cf4d-4409-a6a1-771c731fc6b6 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:31.116463 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ee493247-cf4d-4409-a6a1-771c731fc6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:31.253488 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 924/1851] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:25:31.266406 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b75a510-b116-450f-9963-7bb2d6f7dc4f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:31.270220 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee493247-cf4d-4409-a6a1-771c731fc6b6 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:31.276966 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946eed69-c930-42b5-b7f4-05a5161e31c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:31.278332 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946eed69-c930-42b5-b7f4-05a5161e31c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:31.278905 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946eed69-c930-42b5-b7f4-05a5161e31c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:31.279482 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946eed69-c930-42b5-b7f4-05a5161e31c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:31.281076 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946eed69-c930-42b5-b7f4-05a5161e31c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:31.284520 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee493247-cf4d-4409-a6a1-771c731fc6b6 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:31.285416 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946eed69-c930-42b5-b7f4-05a5161e31c1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:31.296270 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb97bf8-b1a0-40c6-946b-0bff1258121c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:31.297446 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb97bf8-b1a0-40c6-946b-0bff1258121c admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:31.297982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb97bf8-b1a0-40c6-946b-0bff1258121c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:31.298535 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb97bf8-b1a0-40c6-946b-0bff1258121c admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:31.300304 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb97bf8-b1a0-40c6-946b-0bff1258121c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:25:31.300852 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb97bf8-b1a0-40c6-946b-0bff1258121c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:31.302919 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb97bf8-b1a0-40c6-946b-0bff1258121c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:31.311464 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 928/1852] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:25:31.318085 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946eed69-c930-42b5-b7f4-05a5161e31c1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:31.333092 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 925/1853] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:25:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:25:31.334246 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb97bf8-b1a0-40c6-946b-0bff1258121c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:31.346040 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946eed69-c930-42b5-b7f4-05a5161e31c1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:31.353878 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb97bf8-b1a0-40c6-946b-0bff1258121c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:31.356406 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554c4f4-99e2-4de5-ac9d-29ddd9b076a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:31.358186 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554c4f4-99e2-4de5-ac9d-29ddd9b076a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:31.358740 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554c4f4-99e2-4de5-ac9d-29ddd9b076a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:31.359431 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554c4f4-99e2-4de5-ac9d-29ddd9b076a5 admin admin] PATH_INFO: `/v3/projects/37d83134d1c4493a90241f74b348103d/users/7514086938a446cfb360af01aa24b24a/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:31.362431 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2554c4f4-99e2-4de5-ac9d-29ddd9b076a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37d83134d1c4493a90241f74b348103d, user_id=7514086938a446cfb360af01aa24b24a, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:31.365608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed56cc84-490d-404f-9019-c0b576db0da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:31.366300 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2554c4f4-99e2-4de5-ac9d-29ddd9b076a5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:31.367155 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed56cc84-490d-404f-9019-c0b576db0da5 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:31.367641 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed56cc84-490d-404f-9019-c0b576db0da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:31.367825 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed56cc84-490d-404f-9019-c0b576db0da5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:31.368990 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed56cc84-490d-404f-9019-c0b576db0da5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:31.371818 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed56cc84-490d-404f-9019-c0b576db0da5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:31.396315 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 929/1854] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:25:31 2022] PUT /identity/v3/projects/37d83134d1c4493a90241f74b348103d/users/7514086938a446cfb360af01aa24b24a/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:31.413726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572a29b-f569-4d4d-84a4-f1504206d021 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:31.414244 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572a29b-f569-4d4d-84a4-f1504206d021 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:31.414658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572a29b-f569-4d4d-84a4-f1504206d021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:31.431642 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 926/1855] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:25:31 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:25:31.437654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed56cc84-490d-404f-9019-c0b576db0da5 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:31.475333 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c572a29b-f569-4d4d-84a4-f1504206d021 None None] MFA Rules not processed for user `7514086938a446cfb360af01aa24b24a`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:31.476723 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c572a29b-f569-4d4d-84a4-f1504206d021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:31.608001 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed56cc84-490d-404f-9019-c0b576db0da5 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:31.619977 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde94df-9821-46c5-ae00-48102cb5168b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:31.623355 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde94df-9821-46c5-ae00-48102cb5168b admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:31.623355 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde94df-9821-46c5-ae00-48102cb5168b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:31.623355 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde94df-9821-46c5-ae00-48102cb5168b admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:31.624174 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fde94df-9821-46c5-ae00-48102cb5168b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:31.627241 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 930/1856] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:31 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:25:31.627963 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fde94df-9821-46c5-ae00-48102cb5168b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:31.637030 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c572a29b-f569-4d4d-84a4-f1504206d021 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:31.640141 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c572a29b-f569-4d4d-84a4-f1504206d021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:31.692154 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 927/1857] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:25:31 2022] POST /identity/v3/users => generated 290 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:25:31.694494 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde94df-9821-46c5-ae00-48102cb5168b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:31.744653 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde94df-9821-46c5-ae00-48102cb5168b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:31.762678 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9aa0f-2b10-4cb0-8def-305d2e7c2e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:31.763909 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9aa0f-2b10-4cb0-8def-305d2e7c2e16 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:31.764420 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9aa0f-2b10-4cb0-8def-305d2e7c2e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:31.765372 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9aa0f-2b10-4cb0-8def-305d2e7c2e16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:31.766362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9aa0f-2b10-4cb0-8def-305d2e7c2e16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:31.770066 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9aa0f-2b10-4cb0-8def-305d2e7c2e16 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:31.811181 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 928/1858] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:25:31.813342 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9aa0f-2b10-4cb0-8def-305d2e7c2e16 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:31.839495 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9aa0f-2b10-4cb0-8def-305d2e7c2e16 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:31.846406 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c572a29b-f569-4d4d-84a4-f1504206d021 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:31.850834 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c015c38c-f194-4ae7-9a0c-9896cfefaa7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:31.852251 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015c38c-f194-4ae7-9a0c-9896cfefaa7f admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:31.853213 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015c38c-f194-4ae7-9a0c-9896cfefaa7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:31.853850 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c015c38c-f194-4ae7-9a0c-9896cfefaa7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:31.854898 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d31b1-eb60-4fce-af91-ab12ec88ffc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:31.855992 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c015c38c-f194-4ae7-9a0c-9896cfefaa7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:31.856574 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d31b1-eb60-4fce-af91-ab12ec88ffc6 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:31.857072 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d31b1-eb60-4fce-af91-ab12ec88ffc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:31.857711 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d31b1-eb60-4fce-af91-ab12ec88ffc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:31.860132 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c015c38c-f194-4ae7-9a0c-9896cfefaa7f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:31.880276 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:31.881891 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d31b1-eb60-4fce-af91-ab12ec88ffc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:31.888100 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d31b1-eb60-4fce-af91-ab12ec88ffc6 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:31.893801 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 929/1859] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:25:31.906421 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c015c38c-f194-4ae7-9a0c-9896cfefaa7f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:31.913059 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 931/1860] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:25:31 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:25:31.935495 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c015c38c-f194-4ae7-9a0c-9896cfefaa7f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:31.957366 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb49479-e286-4065-a627-43c23a7dd40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:31.959251 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb49479-e286-4065-a627-43c23a7dd40e admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:31.959999 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb49479-e286-4065-a627-43c23a7dd40e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:31.961089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb49479-e286-4065-a627-43c23a7dd40e admin admin] PATH_INFO: `/v3/projects/802cb03e70f14a51ae13ca4dade368f9/users/19e20b73ae814066a184d9784c10dd40/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:32.004363 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb49479-e286-4065-a627-43c23a7dd40e admin admin] RBAC: Authorizing `identity:create_grant(project_id=802cb03e70f14a51ae13ca4dade368f9, user_id=19e20b73ae814066a184d9784c10dd40, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:32.011143 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb49479-e286-4065-a627-43c23a7dd40e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:32.063900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 930/1861] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:25:31 2022] PUT /identity/v3/projects/802cb03e70f14a51ae13ca4dade368f9/users/19e20b73ae814066a184d9784c10dd40/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:32.078657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d110e05-affc-4483-b43d-efa34f65d73c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:32.079235 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d110e05-affc-4483-b43d-efa34f65d73c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:32.080176 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d110e05-affc-4483-b43d-efa34f65d73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:32.136921 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:32.182650 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6d110e05-affc-4483-b43d-efa34f65d73c None None] MFA Rules not processed for user `19e20b73ae814066a184d9784c10dd40`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:32.185908 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6d110e05-affc-4483-b43d-efa34f65d73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:32.437280 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 932/1862] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:25:32.438779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb49479-e286-4065-a627-43c23a7dd40e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:32.464964 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d110e05-affc-4483-b43d-efa34f65d73c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:32.466644 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6d110e05-affc-4483-b43d-efa34f65d73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:32.657940 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb49479-e286-4065-a627-43c23a7dd40e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:32.679176 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca01a38-c229-42f9-a31a-4165e85826fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:32.679835 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca01a38-c229-42f9-a31a-4165e85826fa admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:32.680223 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca01a38-c229-42f9-a31a-4165e85826fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:32.680369 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca01a38-c229-42f9-a31a-4165e85826fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:32.681895 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca01a38-c229-42f9-a31a-4165e85826fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:32.684973 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca01a38-c229-42f9-a31a-4165e85826fa admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:32.693655 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d110e05-affc-4483-b43d-efa34f65d73c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:32.725837 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248275d-39b9-4a3f-b7f8-9acbba9ff942 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:32.726545 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248275d-39b9-4a3f-b7f8-9acbba9ff942 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:32.726858 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248275d-39b9-4a3f-b7f8-9acbba9ff942 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:32.727147 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248275d-39b9-4a3f-b7f8-9acbba9ff942 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:32.738898 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b248275d-39b9-4a3f-b7f8-9acbba9ff942 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:32.788347 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 931/1863] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:25:32 2022] POST /identity/v3/projects => generated 379 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:25:32.789878 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca01a38-c229-42f9-a31a-4165e85826fa admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:32.928396 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b248275d-39b9-4a3f-b7f8-9acbba9ff942 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:32.931502 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b248275d-39b9-4a3f-b7f8-9acbba9ff942 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:32.950491 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 933/1864] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:25:32 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:25:33.047026 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca01a38-c229-42f9-a31a-4165e85826fa admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:33.067393 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846eb672-c43a-4bc9-8bfc-8b3d50b93e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:33.071298 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846eb672-c43a-4bc9-8bfc-8b3d50b93e45 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:33.076309 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846eb672-c43a-4bc9-8bfc-8b3d50b93e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:33.079054 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846eb672-c43a-4bc9-8bfc-8b3d50b93e45 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:33.086288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846eb672-c43a-4bc9-8bfc-8b3d50b93e45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:33.092844 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846eb672-c43a-4bc9-8bfc-8b3d50b93e45 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:33.199543 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 932/1865] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:25:32 2022] POST /identity/v3/users => generated 296 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:25:33.201338 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248275d-39b9-4a3f-b7f8-9acbba9ff942 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:33.223270 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248275d-39b9-4a3f-b7f8-9acbba9ff942 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:33.243276 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57332397-eca7-4832-89ca-c9593a5627ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:33.248282 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57332397-eca7-4832-89ca-c9593a5627ee admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:33.250608 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57332397-eca7-4832-89ca-c9593a5627ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:33.251183 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57332397-eca7-4832-89ca-c9593a5627ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:33.266097 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57332397-eca7-4832-89ca-c9593a5627ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:33.277212 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57332397-eca7-4832-89ca-c9593a5627ee admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:33.311254 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 934/1866] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:25:33.312732 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846eb672-c43a-4bc9-8bfc-8b3d50b93e45 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:33.342638 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846eb672-c43a-4bc9-8bfc-8b3d50b93e45 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:33.373910 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc72ef-62af-47e1-85be-a86ef60b5177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:33.375309 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc72ef-62af-47e1-85be-a86ef60b5177 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:33.375712 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc72ef-62af-47e1-85be-a86ef60b5177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:33.376252 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc72ef-62af-47e1-85be-a86ef60b5177 admin admin] PATH_INFO: `/v3/projects/d094ae1cdf7646b4a1724aedaa6f8d1d/users/9eff4e7f00414b4c9a482a2810dbf745/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:33.429011 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bc72ef-62af-47e1-85be-a86ef60b5177 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d094ae1cdf7646b4a1724aedaa6f8d1d, user_id=9eff4e7f00414b4c9a482a2810dbf745, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:33.437803 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bc72ef-62af-47e1-85be-a86ef60b5177 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:33.496526 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 933/1867] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:25:33 2022] PUT /identity/v3/projects/d094ae1cdf7646b4a1724aedaa6f8d1d/users/9eff4e7f00414b4c9a482a2810dbf745/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:25:33.507399 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57332397-eca7-4832-89ca-c9593a5627ee admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:33.568864 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57332397-eca7-4832-89ca-c9593a5627ee admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:33.706201 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:33.737924 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3aa33-77ed-4948-ac03-c13e79a1fdd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:33.739081 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3aa33-77ed-4948-ac03-c13e79a1fdd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:33.739646 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3aa33-77ed-4948-ac03-c13e79a1fdd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:33.740010 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3aa33-77ed-4948-ac03-c13e79a1fdd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:33.740878 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b3aa33-77ed-4948-ac03-c13e79a1fdd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:33.742640 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b3aa33-77ed-4948-ac03-c13e79a1fdd7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:33.776484 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 935/1868] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:25:33.779985 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc72ef-62af-47e1-85be-a86ef60b5177 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:33.811842 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc72ef-62af-47e1-85be-a86ef60b5177 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:33.818962 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3a574-b3ee-4a1f-8f7c-25337c87597c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:33.822542 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3a574-b3ee-4a1f-8f7c-25337c87597c admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:33.823374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3a574-b3ee-4a1f-8f7c-25337c87597c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:33.824048 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3a574-b3ee-4a1f-8f7c-25337c87597c admin admin] PATH_INFO: `/v3/projects/d094ae1cdf7646b4a1724aedaa6f8d1d/users/9eff4e7f00414b4c9a482a2810dbf745/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:33.827549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c3a574-b3ee-4a1f-8f7c-25337c87597c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d094ae1cdf7646b4a1724aedaa6f8d1d, user_id=9eff4e7f00414b4c9a482a2810dbf745, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:33.832460 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c3a574-b3ee-4a1f-8f7c-25337c87597c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:33.875233 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 934/1869] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:25:33 2022] PUT /identity/v3/projects/d094ae1cdf7646b4a1724aedaa6f8d1d/users/9eff4e7f00414b4c9a482a2810dbf745/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:33.876398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e74fe09-2bd6-4b8c-8907-8293f5be92bc None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:33.876692 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e74fe09-2bd6-4b8c-8907-8293f5be92bc None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:33.877208 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e74fe09-2bd6-4b8c-8907-8293f5be92bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:33.954381 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1e74fe09-2bd6-4b8c-8907-8293f5be92bc None None] MFA Rules not processed for user `9eff4e7f00414b4c9a482a2810dbf745`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:33.955731 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1e74fe09-2bd6-4b8c-8907-8293f5be92bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:34.079106 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 936/1870] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:33 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:25:34.109088 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3a574-b3ee-4a1f-8f7c-25337c87597c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:34.129319 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:34.277867 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3a574-b3ee-4a1f-8f7c-25337c87597c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:34.286908 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f609bf-99b7-4bc0-9093-c3e130522e1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:34.288969 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f609bf-99b7-4bc0-9093-c3e130522e1f service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:34.291850 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f609bf-99b7-4bc0-9093-c3e130522e1f service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:34.293910 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f609bf-99b7-4bc0-9093-c3e130522e1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:34.323165 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f609bf-99b7-4bc0-9093-c3e130522e1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:34.325023 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f609bf-99b7-4bc0-9093-c3e130522e1f service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:34.340447 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 935/1871] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:25:34 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:25:35.059540 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e74fe09-2bd6-4b8c-8907-8293f5be92bc None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:35.077906 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e74fe09-2bd6-4b8c-8907-8293f5be92bc None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:35.087844 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690af536-5bb9-41c0-acb3-a799137cda37 tempest-VolumeQuotaClassesTest-390838890 tempest-VolumeQuotaClassesTest-390838890-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9eff4e7f00414b4c9a482a2810dbf745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd094ae1cdf7646b4a1724aedaa6f8d1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:35.089882 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690af536-5bb9-41c0-acb3-a799137cda37 tempest-VolumeQuotaClassesTest-390838890 tempest-VolumeQuotaClassesTest-390838890-project-admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:35.090664 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690af536-5bb9-41c0-acb3-a799137cda37 tempest-VolumeQuotaClassesTest-390838890 tempest-VolumeQuotaClassesTest-390838890-project-admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:35.091383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690af536-5bb9-41c0-acb3-a799137cda37 tempest-VolumeQuotaClassesTest-390838890 tempest-VolumeQuotaClassesTest-390838890-project-admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:35.093148 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690af536-5bb9-41c0-acb3-a799137cda37 tempest-VolumeQuotaClassesTest-390838890 tempest-VolumeQuotaClassesTest-390838890-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:25:35.093803 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690af536-5bb9-41c0-acb3-a799137cda37 tempest-VolumeQuotaClassesTest-390838890 tempest-VolumeQuotaClassesTest-390838890-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:35.097363 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690af536-5bb9-41c0-acb3-a799137cda37 tempest-VolumeQuotaClassesTest-390838890 tempest-VolumeQuotaClassesTest-390838890-project-admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:35.142263 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 937/1872] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:25:35.144984 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f609bf-99b7-4bc0-9093-c3e130522e1f service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:35.165143 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f609bf-99b7-4bc0-9093-c3e130522e1f service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:35.189066 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d71a0-9a21-4157-a3f6-c145e8a9e190 tempest-VolumeQuotaClassesTest-390838890 tempest-VolumeQuotaClassesTest-390838890-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9eff4e7f00414b4c9a482a2810dbf745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd094ae1cdf7646b4a1724aedaa6f8d1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:35.190878 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d71a0-9a21-4157-a3f6-c145e8a9e190 tempest-VolumeQuotaClassesTest-390838890 tempest-VolumeQuotaClassesTest-390838890-project-admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:35.192265 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d71a0-9a21-4157-a3f6-c145e8a9e190 tempest-VolumeQuotaClassesTest-390838890 tempest-VolumeQuotaClassesTest-390838890-project-admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:35.193197 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d71a0-9a21-4157-a3f6-c145e8a9e190 tempest-VolumeQuotaClassesTest-390838890 tempest-VolumeQuotaClassesTest-390838890-project-admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:35.194905 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d71a0-9a21-4157-a3f6-c145e8a9e190 tempest-VolumeQuotaClassesTest-390838890 tempest-VolumeQuotaClassesTest-390838890-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:35.200017 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d71a0-9a21-4157-a3f6-c145e8a9e190 tempest-VolumeQuotaClassesTest-390838890 tempest-VolumeQuotaClassesTest-390838890-project-admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:35.301444 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 936/1873] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:25:35 2022] POST /identity/v3/projects => generated 354 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:25:35.302960 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690af536-5bb9-41c0-acb3-a799137cda37 tempest-VolumeQuotaClassesTest-390838890 tempest-VolumeQuotaClassesTest-390838890-project-admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:35.355627 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690af536-5bb9-41c0-acb3-a799137cda37 tempest-VolumeQuotaClassesTest-390838890 tempest-VolumeQuotaClassesTest-390838890-project-admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:35.570511 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d610b0-1d76-4e80-9b44-5efa198c1e7d tempest-VolumeQuotaClassesTest-390838890 tempest-VolumeQuotaClassesTest-390838890-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9eff4e7f00414b4c9a482a2810dbf745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd094ae1cdf7646b4a1724aedaa6f8d1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:35.573677 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d610b0-1d76-4e80-9b44-5efa198c1e7d tempest-VolumeQuotaClassesTest-390838890 tempest-VolumeQuotaClassesTest-390838890-project-admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:35.576348 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d610b0-1d76-4e80-9b44-5efa198c1e7d tempest-VolumeQuotaClassesTest-390838890 tempest-VolumeQuotaClassesTest-390838890-project-admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:35.579262 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d610b0-1d76-4e80-9b44-5efa198c1e7d tempest-VolumeQuotaClassesTest-390838890 tempest-VolumeQuotaClassesTest-390838890-project-admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:35.581054 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d610b0-1d76-4e80-9b44-5efa198c1e7d tempest-VolumeQuotaClassesTest-390838890 tempest-VolumeQuotaClassesTest-390838890-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:25:35.581565 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d610b0-1d76-4e80-9b44-5efa198c1e7d tempest-VolumeQuotaClassesTest-390838890 tempest-VolumeQuotaClassesTest-390838890-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:35.591309 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d610b0-1d76-4e80-9b44-5efa198c1e7d tempest-VolumeQuotaClassesTest-390838890 tempest-VolumeQuotaClassesTest-390838890-project-admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:35.636504 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 938/1874] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:25:35.828254 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d71a0-9a21-4157-a3f6-c145e8a9e190 tempest-VolumeQuotaClassesTest-390838890 tempest-VolumeQuotaClassesTest-390838890-project-admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:35.863117 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d71a0-9a21-4157-a3f6-c145e8a9e190 tempest-VolumeQuotaClassesTest-390838890 tempest-VolumeQuotaClassesTest-390838890-project-admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:35.879732 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d63ec-31b7-4c20-a41c-a743a043ca73 tempest-VolumeQuotaClassesTest-390838890 tempest-VolumeQuotaClassesTest-390838890-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9eff4e7f00414b4c9a482a2810dbf745', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd094ae1cdf7646b4a1724aedaa6f8d1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:35.881874 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d63ec-31b7-4c20-a41c-a743a043ca73 tempest-VolumeQuotaClassesTest-390838890 tempest-VolumeQuotaClassesTest-390838890-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:35.882651 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d63ec-31b7-4c20-a41c-a743a043ca73 tempest-VolumeQuotaClassesTest-390838890 tempest-VolumeQuotaClassesTest-390838890-project-admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:35.883343 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d63ec-31b7-4c20-a41c-a743a043ca73 tempest-VolumeQuotaClassesTest-390838890 tempest-VolumeQuotaClassesTest-390838890-project-admin] PATH_INFO: `/v3/projects/750bc0944b7c4ad2bdc501a054016abd` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:35.889068 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43d63ec-31b7-4c20-a41c-a743a043ca73 tempest-VolumeQuotaClassesTest-390838890 tempest-VolumeQuotaClassesTest-390838890-project-admin] RBAC: Authorizing `identity:delete_project(project_id=750bc0944b7c4ad2bdc501a054016abd)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:35.894127 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43d63ec-31b7-4c20-a41c-a743a043ca73 tempest-VolumeQuotaClassesTest-390838890 tempest-VolumeQuotaClassesTest-390838890-project-admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:36.129949 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c43d63ec-31b7-4c20-a41c-a743a043ca73 tempest-VolumeQuotaClassesTest-390838890 tempest-VolumeQuotaClassesTest-390838890-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '750bc0944b7c4ad2bdc501a054016abd', 'request_id': 'req-c43d63ec-31b7-4c20-a41c-a743a043ca73'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:36.183799 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c43d63ec-31b7-4c20-a41c-a743a043ca73 tempest-VolumeQuotaClassesTest-390838890 tempest-VolumeQuotaClassesTest-390838890-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '750bc0944b7c4ad2bdc501a054016abd', 'request_id': 'req-c43d63ec-31b7-4c20-a41c-a743a043ca73'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:36.208606 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c43d63ec-31b7-4c20-a41c-a743a043ca73 tempest-VolumeQuotaClassesTest-390838890 tempest-VolumeQuotaClassesTest-390838890-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '750bc0944b7c4ad2bdc501a054016abd', 'request_id': 'req-c43d63ec-31b7-4c20-a41c-a743a043ca73'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:36.240880 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c43d63ec-31b7-4c20-a41c-a743a043ca73 tempest-VolumeQuotaClassesTest-390838890 tempest-VolumeQuotaClassesTest-390838890-project-admin] The token cache is being invalidate because project 750bc0944b7c4ad2bdc501a054016abd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:25:36.241372 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c43d63ec-31b7-4c20-a41c-a743a043ca73 tempest-VolumeQuotaClassesTest-390838890 tempest-VolumeQuotaClassesTest-390838890-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:36.244416 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c43d63ec-31b7-4c20-a41c-a743a043ca73 tempest-VolumeQuotaClassesTest-390838890 tempest-VolumeQuotaClassesTest-390838890-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:36.246953 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 937/1875] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:25:35 2022] DELETE /identity/v3/projects/750bc0944b7c4ad2bdc501a054016abd => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:36.283031 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dba3d-1a83-4253-8918-e943f2557c10 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:36.284511 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dba3d-1a83-4253-8918-e943f2557c10 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:36.285284 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dba3d-1a83-4253-8918-e943f2557c10 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:36.288212 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 939/1876] 10.222.0.18 () {54 vars in 921 bytes} [Mon Jul 25 19:25:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:25:36.346656 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108732cc-3493-456c-b148-22ec4e5d3e4a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:36.347412 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108732cc-3493-456c-b148-22ec4e5d3e4a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:36.347893 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108732cc-3493-456c-b148-22ec4e5d3e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:36.404532 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-108732cc-3493-456c-b148-22ec4e5d3e4a None None] MFA Rules not processed for user `860e7d26ad364795bb25836aadd4ac7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:36.406451 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-108732cc-3493-456c-b148-22ec4e5d3e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:36.572297 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 938/1877] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Jul 25 19:25:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:25:36.756193 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378dba3d-1a83-4253-8918-e943f2557c10 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:36.759281 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-378dba3d-1a83-4253-8918-e943f2557c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:36.827404 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108732cc-3493-456c-b148-22ec4e5d3e4a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:36.829554 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-108732cc-3493-456c-b148-22ec4e5d3e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:36.848622 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:36.859314 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108732cc-3493-456c-b148-22ec4e5d3e4a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:36.876662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1daaeb2-8a03-45ca-afda-2accf3f40d89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:36.878251 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1daaeb2-8a03-45ca-afda-2accf3f40d89 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:36.878871 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1daaeb2-8a03-45ca-afda-2accf3f40d89 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:36.879689 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1daaeb2-8a03-45ca-afda-2accf3f40d89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:36.882027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:36.897505 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1daaeb2-8a03-45ca-afda-2accf3f40d89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:36.900388 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1daaeb2-8a03-45ca-afda-2accf3f40d89 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:36.912967 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378dba3d-1a83-4253-8918-e943f2557c10 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:36.914093 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 939/1878] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:25:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:25:36.920453 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f369adb-2f53-455e-b917-0e05eda0a266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:36.921879 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f369adb-2f53-455e-b917-0e05eda0a266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:36.922338 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f369adb-2f53-455e-b917-0e05eda0a266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:36.922794 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f369adb-2f53-455e-b917-0e05eda0a266 admin admin] PATH_INFO: `/v3/users/19e20b73ae814066a184d9784c10dd40` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:36.924985 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f369adb-2f53-455e-b917-0e05eda0a266 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19e20b73ae814066a184d9784c10dd40)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:36.928527 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f369adb-2f53-455e-b917-0e05eda0a266 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:37.009738 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3f369adb-2f53-455e-b917-0e05eda0a266 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19e20b73ae814066a184d9784c10dd40'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:37.018572 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3f369adb-2f53-455e-b917-0e05eda0a266 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19e20b73ae814066a184d9784c10dd40'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:37.028482 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3f369adb-2f53-455e-b917-0e05eda0a266 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19e20b73ae814066a184d9784c10dd40'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:37.029173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3f369adb-2f53-455e-b917-0e05eda0a266 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19e20b73ae814066a184d9784c10dd40'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:37.029955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3f369adb-2f53-455e-b917-0e05eda0a266 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19e20b73ae814066a184d9784c10dd40'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:37.055538 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 940/1879] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:25:36 2022] DELETE /identity/v3/users/19e20b73ae814066a184d9784c10dd40 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:37.062778 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5bedc6-9e8e-464a-8910-012449238ca6 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:37.063195 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5bedc6-9e8e-464a-8910-012449238ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:37.063749 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5bedc6-9e8e-464a-8910-012449238ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:37.097344 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3a5bedc6-9e8e-464a-8910-012449238ca6 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:37.099407 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3a5bedc6-9e8e-464a-8910-012449238ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:37.207244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 940/1880] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:25:37.221670 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f369adb-2f53-455e-b917-0e05eda0a266 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:37.223403 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3f369adb-2f53-455e-b917-0e05eda0a266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:37.303678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f369adb-2f53-455e-b917-0e05eda0a266 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:37.307917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52070b04-3cfd-4ceb-bba2-12e8df2d899f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:37.308761 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52070b04-3cfd-4ceb-bba2-12e8df2d899f service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:37.309138 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52070b04-3cfd-4ceb-bba2-12e8df2d899f service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:37.309510 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52070b04-3cfd-4ceb-bba2-12e8df2d899f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:37.321718 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52070b04-3cfd-4ceb-bba2-12e8df2d899f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:37.323324 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52070b04-3cfd-4ceb-bba2-12e8df2d899f service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:37.334959 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 941/1881] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:25:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:25:37.888683 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5bedc6-9e8e-464a-8910-012449238ca6 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:37.891262 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3a5bedc6-9e8e-464a-8910-012449238ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:37.912280 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5bedc6-9e8e-464a-8910-012449238ca6 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:37.924304 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1aa5e-db7c-4a8f-a6be-6687840b8720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:37.925476 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1aa5e-db7c-4a8f-a6be-6687840b8720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:37.925909 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1aa5e-db7c-4a8f-a6be-6687840b8720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:37.926520 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1aa5e-db7c-4a8f-a6be-6687840b8720 admin admin] PATH_INFO: `/v3/projects/802cb03e70f14a51ae13ca4dade368f9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:37.929220 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba1aa5e-db7c-4a8f-a6be-6687840b8720 admin admin] RBAC: Authorizing `identity:delete_project(project_id=802cb03e70f14a51ae13ca4dade368f9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:37.932540 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba1aa5e-db7c-4a8f-a6be-6687840b8720 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:38.010024 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6ba1aa5e-db7c-4a8f-a6be-6687840b8720 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '802cb03e70f14a51ae13ca4dade368f9', 'request_id': 'req-6ba1aa5e-db7c-4a8f-a6be-6687840b8720'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:38.023439 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6ba1aa5e-db7c-4a8f-a6be-6687840b8720 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '802cb03e70f14a51ae13ca4dade368f9', 'request_id': 'req-6ba1aa5e-db7c-4a8f-a6be-6687840b8720'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:38.035744 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6ba1aa5e-db7c-4a8f-a6be-6687840b8720 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '802cb03e70f14a51ae13ca4dade368f9', 'request_id': 'req-6ba1aa5e-db7c-4a8f-a6be-6687840b8720'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:38.059217 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6ba1aa5e-db7c-4a8f-a6be-6687840b8720 admin admin] The token cache is being invalidate because project 802cb03e70f14a51ae13ca4dade368f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:25:38.060029 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6ba1aa5e-db7c-4a8f-a6be-6687840b8720 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:38.061077 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6ba1aa5e-db7c-4a8f-a6be-6687840b8720 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:38.063381 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 941/1882] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:25:37 2022] DELETE /identity/v3/projects/802cb03e70f14a51ae13ca4dade368f9 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:38.068164 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52070b04-3cfd-4ceb-bba2-12e8df2d899f service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:38.082176 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52070b04-3cfd-4ceb-bba2-12e8df2d899f service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:38.173907 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a594f74-f69a-458e-a247-644dbcb0201f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:38.175253 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a594f74-f69a-458e-a247-644dbcb0201f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:38.175603 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a594f74-f69a-458e-a247-644dbcb0201f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:38.175846 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a594f74-f69a-458e-a247-644dbcb0201f admin admin] PATH_INFO: `/v3/users/9eff4e7f00414b4c9a482a2810dbf745` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:38.177498 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a594f74-f69a-458e-a247-644dbcb0201f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eff4e7f00414b4c9a482a2810dbf745)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:38.179560 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a594f74-f69a-458e-a247-644dbcb0201f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:38.188377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:38.286630 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3a594f74-f69a-458e-a247-644dbcb0201f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eff4e7f00414b4c9a482a2810dbf745'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:38.310320 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3a594f74-f69a-458e-a247-644dbcb0201f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eff4e7f00414b4c9a482a2810dbf745'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:38.324256 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3a594f74-f69a-458e-a247-644dbcb0201f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eff4e7f00414b4c9a482a2810dbf745'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:38.325043 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3a594f74-f69a-458e-a247-644dbcb0201f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eff4e7f00414b4c9a482a2810dbf745'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:38.325802 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3a594f74-f69a-458e-a247-644dbcb0201f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eff4e7f00414b4c9a482a2810dbf745'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:38.349669 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 942/1883] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:25:38 2022] DELETE /identity/v3/users/9eff4e7f00414b4c9a482a2810dbf745 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:38.910758 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1aa5e-db7c-4a8f-a6be-6687840b8720 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:38.913432 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6ba1aa5e-db7c-4a8f-a6be-6687840b8720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:39.004169 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1aa5e-db7c-4a8f-a6be-6687840b8720 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:39.224644 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534ad30-097b-4a3a-a798-6dcc45cea1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:39.226286 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534ad30-097b-4a3a-a798-6dcc45cea1eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:39.226908 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534ad30-097b-4a3a-a798-6dcc45cea1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:39.227541 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534ad30-097b-4a3a-a798-6dcc45cea1eb admin admin] PATH_INFO: `/v3/projects/d094ae1cdf7646b4a1724aedaa6f8d1d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:39.231032 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9534ad30-097b-4a3a-a798-6dcc45cea1eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d094ae1cdf7646b4a1724aedaa6f8d1d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:39.235198 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9534ad30-097b-4a3a-a798-6dcc45cea1eb admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:39.386948 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:39.570838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9534ad30-097b-4a3a-a798-6dcc45cea1eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd094ae1cdf7646b4a1724aedaa6f8d1d', 'request_id': 'req-9534ad30-097b-4a3a-a798-6dcc45cea1eb'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:39.610355 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9534ad30-097b-4a3a-a798-6dcc45cea1eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd094ae1cdf7646b4a1724aedaa6f8d1d', 'request_id': 'req-9534ad30-097b-4a3a-a798-6dcc45cea1eb'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:39.625939 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9534ad30-097b-4a3a-a798-6dcc45cea1eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd094ae1cdf7646b4a1724aedaa6f8d1d', 'request_id': 'req-9534ad30-097b-4a3a-a798-6dcc45cea1eb'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:39.664455 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9534ad30-097b-4a3a-a798-6dcc45cea1eb admin admin] The token cache is being invalidate because project d094ae1cdf7646b4a1724aedaa6f8d1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:25:39.665312 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9534ad30-097b-4a3a-a798-6dcc45cea1eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:39.667728 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9534ad30-097b-4a3a-a798-6dcc45cea1eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:39.679186 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 942/1884] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:25:38 2022] DELETE /identity/v3/projects/d094ae1cdf7646b4a1724aedaa6f8d1d => generated 0 bytes in 769 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:39.685378 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b642154-316b-4991-960e-a51a93f744eb None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:39.686343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b642154-316b-4991-960e-a51a93f744eb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:39.686343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b642154-316b-4991-960e-a51a93f744eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:39.741462 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1b642154-316b-4991-960e-a51a93f744eb None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:39.745233 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1b642154-316b-4991-960e-a51a93f744eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:39.955035 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 943/1885] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:25:39.965969 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84172bb1-141c-4362-b285-19bfebdd3444 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:39.966987 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84172bb1-141c-4362-b285-19bfebdd3444 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:39.967462 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84172bb1-141c-4362-b285-19bfebdd3444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:40.062665 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-84172bb1-141c-4362-b285-19bfebdd3444 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:40.066083 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-84172bb1-141c-4362-b285-19bfebdd3444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:40.092246 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 943/1886] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:25:40.097058 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b642154-316b-4991-960e-a51a93f744eb None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:40.147142 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b642154-316b-4991-960e-a51a93f744eb None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:40.193126 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c5449-674c-473d-b0ff-c72945564046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:40.213809 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c5449-674c-473d-b0ff-c72945564046 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:40.220711 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c5449-674c-473d-b0ff-c72945564046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:40.221534 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c5449-674c-473d-b0ff-c72945564046 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:40.223241 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c5449-674c-473d-b0ff-c72945564046 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:25:40.224107 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c5449-674c-473d-b0ff-c72945564046 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:40.228668 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c5449-674c-473d-b0ff-c72945564046 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:40.345531 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 944/1887] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:25:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:25:40.350928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84172bb1-141c-4362-b285-19bfebdd3444 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:40.407549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84172bb1-141c-4362-b285-19bfebdd3444 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:40.462775 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609dddca-f5a8-4a6d-b1c0-e00f079cd17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:40.474638 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609dddca-f5a8-4a6d-b1c0-e00f079cd17b admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:40.476094 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609dddca-f5a8-4a6d-b1c0-e00f079cd17b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:40.480210 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609dddca-f5a8-4a6d-b1c0-e00f079cd17b admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:40.499457 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609dddca-f5a8-4a6d-b1c0-e00f079cd17b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:40.518140 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609dddca-f5a8-4a6d-b1c0-e00f079cd17b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:40.624216 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 944/1888] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:25:40 2022] POST /identity/v3/projects => generated 387 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:25:40.628988 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c5449-674c-473d-b0ff-c72945564046 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:40.683661 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c5449-674c-473d-b0ff-c72945564046 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:40.899171 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06a69eb-faa6-47c4-94fc-6332a4fe920f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:40.902123 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a69eb-faa6-47c4-94fc-6332a4fe920f admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:40.902951 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a69eb-faa6-47c4-94fc-6332a4fe920f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:40.903774 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a69eb-faa6-47c4-94fc-6332a4fe920f admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:40.905455 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06a69eb-faa6-47c4-94fc-6332a4fe920f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:40.910440 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06a69eb-faa6-47c4-94fc-6332a4fe920f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:40.977374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609dddca-f5a8-4a6d-b1c0-e00f079cd17b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:40.988934 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 945/1889] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:25:40 2022] POST /identity/v3/users => generated 294 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:25:41.006804 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609dddca-f5a8-4a6d-b1c0-e00f079cd17b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:41.034381 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a641b3-2f7c-492f-b929-b8b2eebc47be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:41.035167 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a641b3-2f7c-492f-b929-b8b2eebc47be admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:41.035646 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a641b3-2f7c-492f-b929-b8b2eebc47be admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:41.035982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a641b3-2f7c-492f-b929-b8b2eebc47be admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:41.039880 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a641b3-2f7c-492f-b929-b8b2eebc47be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:41.042154 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a641b3-2f7c-492f-b929-b8b2eebc47be admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:41.127294 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 945/1890] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:25:41.129361 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06a69eb-faa6-47c4-94fc-6332a4fe920f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:41.154288 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:41.181681 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06a69eb-faa6-47c4-94fc-6332a4fe920f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:41.209576 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71dcbff-e8c8-4729-87ac-669c0d2d787d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:41.211732 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71dcbff-e8c8-4729-87ac-669c0d2d787d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:41.212869 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71dcbff-e8c8-4729-87ac-669c0d2d787d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:41.213625 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71dcbff-e8c8-4729-87ac-669c0d2d787d admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:41.219375 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71dcbff-e8c8-4729-87ac-669c0d2d787d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:41.229805 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71dcbff-e8c8-4729-87ac-669c0d2d787d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:41.304930 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 946/1891] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:25:41.309889 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a641b3-2f7c-492f-b929-b8b2eebc47be admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:41.342126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a641b3-2f7c-492f-b929-b8b2eebc47be admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:41.368430 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc67593f-b851-4616-a9b1-e5876378657c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:41.373747 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc67593f-b851-4616-a9b1-e5876378657c admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:41.377117 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc67593f-b851-4616-a9b1-e5876378657c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:41.378848 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc67593f-b851-4616-a9b1-e5876378657c admin admin] PATH_INFO: `/v3/projects/48bff963d24d4380a384d2466ddca9a1/users/77b9784cd8ca4026aaa73e3b0ad2f1f7/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:41.476185 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc67593f-b851-4616-a9b1-e5876378657c admin admin] RBAC: Authorizing `identity:create_grant(project_id=48bff963d24d4380a384d2466ddca9a1, user_id=77b9784cd8ca4026aaa73e3b0ad2f1f7, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:41.490057 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc67593f-b851-4616-a9b1-e5876378657c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:41.552829 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 946/1892] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:25:41 2022] PUT /identity/v3/projects/48bff963d24d4380a384d2466ddca9a1/users/77b9784cd8ca4026aaa73e3b0ad2f1f7/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:41.572944 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cbfbd1-6f03-4c4a-a385-7ccd70333a03 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:41.573634 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cbfbd1-6f03-4c4a-a385-7ccd70333a03 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:41.574190 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cbfbd1-6f03-4c4a-a385-7ccd70333a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:41.621365 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-02cbfbd1-6f03-4c4a-a385-7ccd70333a03 None None] MFA Rules not processed for user `77b9784cd8ca4026aaa73e3b0ad2f1f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:41.623181 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-02cbfbd1-6f03-4c4a-a385-7ccd70333a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:41.751930 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 947/1893] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:25:41.752774 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc67593f-b851-4616-a9b1-e5876378657c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:41.912132 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc67593f-b851-4616-a9b1-e5876378657c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:41.922956 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788cc80b-1e0d-4b36-9ed4-2170824efe40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:41.924175 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788cc80b-1e0d-4b36-9ed4-2170824efe40 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:41.924633 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788cc80b-1e0d-4b36-9ed4-2170824efe40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:41.939770 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788cc80b-1e0d-4b36-9ed4-2170824efe40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:41.941196 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788cc80b-1e0d-4b36-9ed4-2170824efe40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:41.945151 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788cc80b-1e0d-4b36-9ed4-2170824efe40 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:42.001435 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 947/1894] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:25:41 2022] POST /identity/v3/projects => generated 385 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:25:42.003878 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cbfbd1-6f03-4c4a-a385-7ccd70333a03 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:42.031885 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cbfbd1-6f03-4c4a-a385-7ccd70333a03 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:42.169846 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d59c7f3-6559-4ad9-b659-62e673c405a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:42.170564 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59c7f3-6559-4ad9-b659-62e673c405a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:42.170851 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59c7f3-6559-4ad9-b659-62e673c405a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:42.171135 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59c7f3-6559-4ad9-b659-62e673c405a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:42.175123 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d59c7f3-6559-4ad9-b659-62e673c405a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:42.177044 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d59c7f3-6559-4ad9-b659-62e673c405a0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:42.215660 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 948/1895] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:25:42 2022] POST /identity/v3/users => generated 299 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:25:42.220235 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788cc80b-1e0d-4b36-9ed4-2170824efe40 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:42.232190 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788cc80b-1e0d-4b36-9ed4-2170824efe40 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:42.239776 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc0225-b5b7-4f53-8733-2458f4df2dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:42.240595 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc0225-b5b7-4f53-8733-2458f4df2dfd admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:42.241066 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc0225-b5b7-4f53-8733-2458f4df2dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:42.241523 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc0225-b5b7-4f53-8733-2458f4df2dfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:42.242827 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc0225-b5b7-4f53-8733-2458f4df2dfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:42.245996 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc0225-b5b7-4f53-8733-2458f4df2dfd admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:42.273535 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 948/1896] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:25:42 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:25:42.279387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d59c7f3-6559-4ad9-b659-62e673c405a0 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:42.295292 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d59c7f3-6559-4ad9-b659-62e673c405a0 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:42.301635 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e0f5b-676d-435c-ab77-f18082d74956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:42.302864 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e0f5b-676d-435c-ab77-f18082d74956 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:42.303327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e0f5b-676d-435c-ab77-f18082d74956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:42.303813 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e0f5b-676d-435c-ab77-f18082d74956 admin admin] PATH_INFO: `/v3/projects/9203cb7553af4ba19fcef0ab41e37378/users/586a192ffd3f4778b43194effcbe2982/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:42.380323 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8e0f5b-676d-435c-ab77-f18082d74956 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9203cb7553af4ba19fcef0ab41e37378, user_id=586a192ffd3f4778b43194effcbe2982, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:42.397236 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8e0f5b-676d-435c-ab77-f18082d74956 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:42.446501 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 949/1897] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:25:42 2022] PUT /identity/v3/projects/9203cb7553af4ba19fcef0ab41e37378/users/586a192ffd3f4778b43194effcbe2982/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:25:42.451459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc0225-b5b7-4f53-8733-2458f4df2dfd admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:42.476250 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc0225-b5b7-4f53-8733-2458f4df2dfd admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:42.505059 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:42.649144 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e3030-7faf-41c8-b014-51d2b783b876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:42.651138 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e3030-7faf-41c8-b014-51d2b783b876 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:42.651752 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e3030-7faf-41c8-b014-51d2b783b876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:42.652357 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e3030-7faf-41c8-b014-51d2b783b876 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:42.654153 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e3030-7faf-41c8-b014-51d2b783b876 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:42.657686 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e3030-7faf-41c8-b014-51d2b783b876 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:42.679830 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 949/1898] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:25:42 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:25:42.685585 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e0f5b-676d-435c-ab77-f18082d74956 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:42.703204 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e0f5b-676d-435c-ab77-f18082d74956 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:42.710599 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c6dfe-f91f-452e-aba3-727d17c96e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:42.712926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c6dfe-f91f-452e-aba3-727d17c96e8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:42.713471 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c6dfe-f91f-452e-aba3-727d17c96e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:42.713910 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c6dfe-f91f-452e-aba3-727d17c96e8c admin admin] PATH_INFO: `/v3/projects/9203cb7553af4ba19fcef0ab41e37378/users/586a192ffd3f4778b43194effcbe2982/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:42.717955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c6dfe-f91f-452e-aba3-727d17c96e8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9203cb7553af4ba19fcef0ab41e37378, user_id=586a192ffd3f4778b43194effcbe2982, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:42.723579 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c6dfe-f91f-452e-aba3-727d17c96e8c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:42.739223 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 950/1899] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:25:42 2022] PUT /identity/v3/projects/9203cb7553af4ba19fcef0ab41e37378/users/586a192ffd3f4778b43194effcbe2982/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:42.753223 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122ca9e-717a-4f8e-b834-4d16f5cb2b72 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:42.753647 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122ca9e-717a-4f8e-b834-4d16f5cb2b72 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:42.754058 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122ca9e-717a-4f8e-b834-4d16f5cb2b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:42.810260 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b122ca9e-717a-4f8e-b834-4d16f5cb2b72 None None] MFA Rules not processed for user `586a192ffd3f4778b43194effcbe2982`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:42.812221 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b122ca9e-717a-4f8e-b834-4d16f5cb2b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:42.917294 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 950/1900] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:42 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:25:42.930476 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c6dfe-f91f-452e-aba3-727d17c96e8c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:42.934006 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-824c6dfe-f91f-452e-aba3-727d17c96e8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:43.021205 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:43.064407 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c6dfe-f91f-452e-aba3-727d17c96e8c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:43.071776 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972238c-addf-4467-9a1e-5a5042ddd8b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:43.072907 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972238c-addf-4467-9a1e-5a5042ddd8b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:43.073208 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972238c-addf-4467-9a1e-5a5042ddd8b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:43.073494 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972238c-addf-4467-9a1e-5a5042ddd8b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:43.086345 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972238c-addf-4467-9a1e-5a5042ddd8b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:43.088498 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972238c-addf-4467-9a1e-5a5042ddd8b5 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:43.100232 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 951/1901] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:25:42 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:25:44.334212 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b122ca9e-717a-4f8e-b834-4d16f5cb2b72 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:44.471434 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b122ca9e-717a-4f8e-b834-4d16f5cb2b72 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:44.482162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b369b-31ee-4fa3-b90a-189c445c15a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:44.483219 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b369b-31ee-4fa3-b90a-189c445c15a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:44.483640 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b369b-31ee-4fa3-b90a-189c445c15a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:44.484085 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b369b-31ee-4fa3-b90a-189c445c15a4 admin admin] PATH_INFO: `/v3/users/77b9784cd8ca4026aaa73e3b0ad2f1f7` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:44.486243 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b369b-31ee-4fa3-b90a-189c445c15a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77b9784cd8ca4026aaa73e3b0ad2f1f7)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:44.493062 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b369b-31ee-4fa3-b90a-189c445c15a4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:44.610626 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:44.641688 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3f9b369b-31ee-4fa3-b90a-189c445c15a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77b9784cd8ca4026aaa73e3b0ad2f1f7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:44.693346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3f9b369b-31ee-4fa3-b90a-189c445c15a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77b9784cd8ca4026aaa73e3b0ad2f1f7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:44.707410 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3f9b369b-31ee-4fa3-b90a-189c445c15a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77b9784cd8ca4026aaa73e3b0ad2f1f7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:44.708199 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3f9b369b-31ee-4fa3-b90a-189c445c15a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77b9784cd8ca4026aaa73e3b0ad2f1f7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:44.743193 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3f9b369b-31ee-4fa3-b90a-189c445c15a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77b9784cd8ca4026aaa73e3b0ad2f1f7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:44.771609 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 951/1902] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:25:44 2022] DELETE /identity/v3/users/77b9784cd8ca4026aaa73e3b0ad2f1f7 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:44.779616 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec38e7-cb95-4b89-a781-264730f37e16 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:44.781476 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec38e7-cb95-4b89-a781-264730f37e16 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:44.782202 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec38e7-cb95-4b89-a781-264730f37e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:44.850687 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f2ec38e7-cb95-4b89-a781-264730f37e16 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:44.853737 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f2ec38e7-cb95-4b89-a781-264730f37e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:45.069552 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 952/1903] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:25:45.083871 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b369b-31ee-4fa3-b90a-189c445c15a4 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:45.085533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3f9b369b-31ee-4fa3-b90a-189c445c15a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:45.239389 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b369b-31ee-4fa3-b90a-189c445c15a4 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:45.245568 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa8041-4c4f-4e0d-86a6-845f35a34313 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:45.246457 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa8041-4c4f-4e0d-86a6-845f35a34313 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:45.246818 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa8041-4c4f-4e0d-86a6-845f35a34313 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:45.247068 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa8041-4c4f-4e0d-86a6-845f35a34313 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:45.267020 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa8041-4c4f-4e0d-86a6-845f35a34313 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:45.271268 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa8041-4c4f-4e0d-86a6-845f35a34313 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:45.296339 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 952/1904] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:25:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:25:45.966804 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec38e7-cb95-4b89-a781-264730f37e16 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:45.970606 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f2ec38e7-cb95-4b89-a781-264730f37e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:46.027457 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec38e7-cb95-4b89-a781-264730f37e16 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:46.034835 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d564be2-95c7-4d03-99f1-159fc21d20ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:46.036520 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d564be2-95c7-4d03-99f1-159fc21d20ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:46.037013 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d564be2-95c7-4d03-99f1-159fc21d20ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:46.037389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d564be2-95c7-4d03-99f1-159fc21d20ab admin admin] PATH_INFO: `/v3/projects/48bff963d24d4380a384d2466ddca9a1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:46.038970 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d564be2-95c7-4d03-99f1-159fc21d20ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=48bff963d24d4380a384d2466ddca9a1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:46.042860 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d564be2-95c7-4d03-99f1-159fc21d20ab admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:46.139345 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8d564be2-95c7-4d03-99f1-159fc21d20ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48bff963d24d4380a384d2466ddca9a1', 'request_id': 'req-8d564be2-95c7-4d03-99f1-159fc21d20ab'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:46.152949 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8d564be2-95c7-4d03-99f1-159fc21d20ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48bff963d24d4380a384d2466ddca9a1', 'request_id': 'req-8d564be2-95c7-4d03-99f1-159fc21d20ab'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:46.162086 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8d564be2-95c7-4d03-99f1-159fc21d20ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48bff963d24d4380a384d2466ddca9a1', 'request_id': 'req-8d564be2-95c7-4d03-99f1-159fc21d20ab'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:46.197552 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8d564be2-95c7-4d03-99f1-159fc21d20ab admin admin] The token cache is being invalidate because project 48bff963d24d4380a384d2466ddca9a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:25:46.198733 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8d564be2-95c7-4d03-99f1-159fc21d20ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:46.200215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8d564be2-95c7-4d03-99f1-159fc21d20ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:46.203695 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 953/1905] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:25:45 2022] DELETE /identity/v3/projects/48bff963d24d4380a384d2466ddca9a1 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:46.208539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa8041-4c4f-4e0d-86a6-845f35a34313 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:46.229326 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa8041-4c4f-4e0d-86a6-845f35a34313 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:46.363607 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94660bd-275e-4930-8b9d-e4950b764fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:46.365454 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94660bd-275e-4930-8b9d-e4950b764fa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:46.366340 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94660bd-275e-4930-8b9d-e4950b764fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:46.367070 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94660bd-275e-4930-8b9d-e4950b764fa8 admin admin] PATH_INFO: `/v3/users/586a192ffd3f4778b43194effcbe2982` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:46.373589 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94660bd-275e-4930-8b9d-e4950b764fa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=586a192ffd3f4778b43194effcbe2982)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:46.382875 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94660bd-275e-4930-8b9d-e4950b764fa8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:46.499333 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b94660bd-275e-4930-8b9d-e4950b764fa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '586a192ffd3f4778b43194effcbe2982'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:46.522243 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b94660bd-275e-4930-8b9d-e4950b764fa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '586a192ffd3f4778b43194effcbe2982'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:46.537437 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:46.550033 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b94660bd-275e-4930-8b9d-e4950b764fa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '586a192ffd3f4778b43194effcbe2982'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:46.552589 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b94660bd-275e-4930-8b9d-e4950b764fa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '586a192ffd3f4778b43194effcbe2982'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:46.596060 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b94660bd-275e-4930-8b9d-e4950b764fa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '586a192ffd3f4778b43194effcbe2982'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:46.610474 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 953/1906] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:25:46 2022] DELETE /identity/v3/users/586a192ffd3f4778b43194effcbe2982 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:47.158243 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d564be2-95c7-4d03-99f1-159fc21d20ab admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:47.161815 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8d564be2-95c7-4d03-99f1-159fc21d20ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:47.221633 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d564be2-95c7-4d03-99f1-159fc21d20ab admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:47.333001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1658d654-54ad-4e01-bfb9-1ee0309b4bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:47.334313 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658d654-54ad-4e01-bfb9-1ee0309b4bee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:47.334790 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658d654-54ad-4e01-bfb9-1ee0309b4bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:47.335249 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658d654-54ad-4e01-bfb9-1ee0309b4bee admin admin] PATH_INFO: `/v3/projects/9203cb7553af4ba19fcef0ab41e37378` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:47.338281 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1658d654-54ad-4e01-bfb9-1ee0309b4bee admin admin] RBAC: Authorizing `identity:delete_project(project_id=9203cb7553af4ba19fcef0ab41e37378)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:47.342124 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1658d654-54ad-4e01-bfb9-1ee0309b4bee admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:47.344986 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:47.434851 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1658d654-54ad-4e01-bfb9-1ee0309b4bee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9203cb7553af4ba19fcef0ab41e37378', 'request_id': 'req-1658d654-54ad-4e01-bfb9-1ee0309b4bee'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:47.442603 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1658d654-54ad-4e01-bfb9-1ee0309b4bee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9203cb7553af4ba19fcef0ab41e37378', 'request_id': 'req-1658d654-54ad-4e01-bfb9-1ee0309b4bee'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:47.454074 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1658d654-54ad-4e01-bfb9-1ee0309b4bee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9203cb7553af4ba19fcef0ab41e37378', 'request_id': 'req-1658d654-54ad-4e01-bfb9-1ee0309b4bee'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:47.478539 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1658d654-54ad-4e01-bfb9-1ee0309b4bee admin admin] The token cache is being invalidate because project 9203cb7553af4ba19fcef0ab41e37378 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:25:47.479065 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1658d654-54ad-4e01-bfb9-1ee0309b4bee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:47.480008 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1658d654-54ad-4e01-bfb9-1ee0309b4bee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:47.481762 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 954/1907] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:25:47 2022] DELETE /identity/v3/projects/9203cb7553af4ba19fcef0ab41e37378 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:47.488371 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5abada-abe3-4f68-9787-2244593efa47 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:47.489042 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5abada-abe3-4f68-9787-2244593efa47 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:47.489042 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5abada-abe3-4f68-9787-2244593efa47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:47.513280 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5e5abada-abe3-4f68-9787-2244593efa47 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:47.514651 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5e5abada-abe3-4f68-9787-2244593efa47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:47.592481 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 954/1908] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:25:47.602162 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95bbe2-7c04-49aa-8ebf-26d5a1e4c6b1 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:47.602675 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95bbe2-7c04-49aa-8ebf-26d5a1e4c6b1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:47.603157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95bbe2-7c04-49aa-8ebf-26d5a1e4c6b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:47.636238 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-fa95bbe2-7c04-49aa-8ebf-26d5a1e4c6b1 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:47.638467 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fa95bbe2-7c04-49aa-8ebf-26d5a1e4c6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:47.657480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 955/1909] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:25:47.659920 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5abada-abe3-4f68-9787-2244593efa47 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:47.676858 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5abada-abe3-4f68-9787-2244593efa47 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:47.684985 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e848eb4-3e7f-42f5-ace3-448e52c6de01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:47.686107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e848eb4-3e7f-42f5-ace3-448e52c6de01 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:47.686310 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e848eb4-3e7f-42f5-ace3-448e52c6de01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:47.686549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e848eb4-3e7f-42f5-ace3-448e52c6de01 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:47.687314 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e848eb4-3e7f-42f5-ace3-448e52c6de01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:25:47.687743 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e848eb4-3e7f-42f5-ace3-448e52c6de01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:47.689412 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e848eb4-3e7f-42f5-ace3-448e52c6de01 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:47.705257 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 955/1910] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:25:47.709390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95bbe2-7c04-49aa-8ebf-26d5a1e4c6b1 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:47.721701 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95bbe2-7c04-49aa-8ebf-26d5a1e4c6b1 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:47.732249 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc8c04-c062-4fad-bdbf-520732c55f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:47.733464 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc8c04-c062-4fad-bdbf-520732c55f37 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:47.733987 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc8c04-c062-4fad-bdbf-520732c55f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:47.734485 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc8c04-c062-4fad-bdbf-520732c55f37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:47.735454 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cc8c04-c062-4fad-bdbf-520732c55f37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:47.739445 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cc8c04-c062-4fad-bdbf-520732c55f37 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:47.795095 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 956/1911] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:25:47 2022] POST /identity/v3/projects => generated 383 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:25:47.799961 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e848eb4-3e7f-42f5-ace3-448e52c6de01 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:47.813272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e848eb4-3e7f-42f5-ace3-448e52c6de01 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:47.941743 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67a16f-35ad-4665-9ec2-cd9e645e1c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:47.942990 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67a16f-35ad-4665-9ec2-cd9e645e1c92 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:47.943490 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67a16f-35ad-4665-9ec2-cd9e645e1c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:47.944046 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67a16f-35ad-4665-9ec2-cd9e645e1c92 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:47.945810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa67a16f-35ad-4665-9ec2-cd9e645e1c92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:47.950512 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa67a16f-35ad-4665-9ec2-cd9e645e1c92 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:47.996990 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 956/1912] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:25:47 2022] POST /identity/v3/users => generated 292 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:25:47.999001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc8c04-c062-4fad-bdbf-520732c55f37 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:48.014127 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc8c04-c062-4fad-bdbf-520732c55f37 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:48.021930 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b41b80-7d63-4ca8-9f10-f0809b6e1275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:48.022631 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b41b80-7d63-4ca8-9f10-f0809b6e1275 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:48.022920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b41b80-7d63-4ca8-9f10-f0809b6e1275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:48.023199 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b41b80-7d63-4ca8-9f10-f0809b6e1275 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:48.024057 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b41b80-7d63-4ca8-9f10-f0809b6e1275 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:48.027551 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b41b80-7d63-4ca8-9f10-f0809b6e1275 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:48.049672 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 957/1913] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:25:47 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:25:48.050918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67a16f-35ad-4665-9ec2-cd9e645e1c92 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:48.065439 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67a16f-35ad-4665-9ec2-cd9e645e1c92 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:48.072736 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d20e77-1f67-4d2e-9c4f-c4bba6b89d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:48.073479 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d20e77-1f67-4d2e-9c4f-c4bba6b89d7c admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:48.073770 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d20e77-1f67-4d2e-9c4f-c4bba6b89d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:48.074054 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d20e77-1f67-4d2e-9c4f-c4bba6b89d7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:48.074807 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d20e77-1f67-4d2e-9c4f-c4bba6b89d7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:48.076527 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d20e77-1f67-4d2e-9c4f-c4bba6b89d7c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:48.095003 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 957/1914] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:25:48.101826 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b41b80-7d63-4ca8-9f10-f0809b6e1275 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:48.123139 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b41b80-7d63-4ca8-9f10-f0809b6e1275 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:48.134337 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3c40a-d79a-44a7-93ac-d57c8f97461b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:48.134688 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3c40a-d79a-44a7-93ac-d57c8f97461b admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:48.135015 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3c40a-d79a-44a7-93ac-d57c8f97461b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:48.135548 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3c40a-d79a-44a7-93ac-d57c8f97461b admin admin] PATH_INFO: `/v3/projects/88dcf6e9e51c44cc9cabb50d37c5e813/users/938ac4f195cd466199de039650614012/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:48.170016 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd3c40a-d79a-44a7-93ac-d57c8f97461b admin admin] RBAC: Authorizing `identity:create_grant(project_id=88dcf6e9e51c44cc9cabb50d37c5e813, user_id=938ac4f195cd466199de039650614012, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:48.174940 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd3c40a-d79a-44a7-93ac-d57c8f97461b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:48.214868 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 958/1915] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:25:48 2022] PUT /identity/v3/projects/88dcf6e9e51c44cc9cabb50d37c5e813/users/938ac4f195cd466199de039650614012/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:48.234524 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba93e83f-13a4-48eb-916d-6d5039ce45e5 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:48.235025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba93e83f-13a4-48eb-916d-6d5039ce45e5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:48.235538 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba93e83f-13a4-48eb-916d-6d5039ce45e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:48.298600 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ba93e83f-13a4-48eb-916d-6d5039ce45e5 None None] MFA Rules not processed for user `938ac4f195cd466199de039650614012`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:48.301249 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ba93e83f-13a4-48eb-916d-6d5039ce45e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:48.374206 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:48.444308 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 958/1916] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:25:48.466104 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3c40a-d79a-44a7-93ac-d57c8f97461b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:48.470051 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fcd3c40a-d79a-44a7-93ac-d57c8f97461b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:48.590977 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3c40a-d79a-44a7-93ac-d57c8f97461b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:48.596723 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea600a2-d02f-41e9-bacc-40bce6861470 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:48.597472 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea600a2-d02f-41e9-bacc-40bce6861470 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:48.597859 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea600a2-d02f-41e9-bacc-40bce6861470 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:48.598201 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea600a2-d02f-41e9-bacc-40bce6861470 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:48.613749 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea600a2-d02f-41e9-bacc-40bce6861470 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:48.616584 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea600a2-d02f-41e9-bacc-40bce6861470 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:48.634062 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 959/1917] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:25:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:25:55.765741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba93e83f-13a4-48eb-916d-6d5039ce45e5 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:55.767473 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ba93e83f-13a4-48eb-916d-6d5039ce45e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:56.021177 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba93e83f-13a4-48eb-916d-6d5039ce45e5 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:56.066096 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23674b36-0ec2-4ea1-89db-b0dc8963276c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:56.069719 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23674b36-0ec2-4ea1-89db-b0dc8963276c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:56.070639 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23674b36-0ec2-4ea1-89db-b0dc8963276c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:56.072118 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23674b36-0ec2-4ea1-89db-b0dc8963276c admin admin] PATH_INFO: `/v3/users/f63354875bca405199ae42ce032743fa` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:56.079159 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23674b36-0ec2-4ea1-89db-b0dc8963276c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f63354875bca405199ae42ce032743fa)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:56.085236 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23674b36-0ec2-4ea1-89db-b0dc8963276c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:56.278903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-23674b36-0ec2-4ea1-89db-b0dc8963276c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f63354875bca405199ae42ce032743fa'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:56.294795 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-23674b36-0ec2-4ea1-89db-b0dc8963276c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f63354875bca405199ae42ce032743fa'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:56.316037 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-23674b36-0ec2-4ea1-89db-b0dc8963276c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f63354875bca405199ae42ce032743fa'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:56.318122 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-23674b36-0ec2-4ea1-89db-b0dc8963276c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f63354875bca405199ae42ce032743fa'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:56.351449 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-23674b36-0ec2-4ea1-89db-b0dc8963276c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f63354875bca405199ae42ce032743fa'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:56.372684 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 959/1918] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:25:55 2022] DELETE /identity/v3/users/f63354875bca405199ae42ce032743fa => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:56.376384 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df3716-a4c5-4934-94c8-11ad67c2e6cf None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:56.377162 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df3716-a4c5-4934-94c8-11ad67c2e6cf None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:56.380015 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df3716-a4c5-4934-94c8-11ad67c2e6cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:56.458842 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b2df3716-a4c5-4934-94c8-11ad67c2e6cf None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:56.461341 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b2df3716-a4c5-4934-94c8-11ad67c2e6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:56.595777 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:56.751656 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 960/1919] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:25:56.769759 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23674b36-0ec2-4ea1-89db-b0dc8963276c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:56.772218 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-23674b36-0ec2-4ea1-89db-b0dc8963276c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:56.919684 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23674b36-0ec2-4ea1-89db-b0dc8963276c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:56.927292 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af496a4a-0e95-4806-9fbd-892ce569be25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:56.930695 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af496a4a-0e95-4806-9fbd-892ce569be25 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:56.930695 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af496a4a-0e95-4806-9fbd-892ce569be25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:56.930695 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af496a4a-0e95-4806-9fbd-892ce569be25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:56.949957 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af496a4a-0e95-4806-9fbd-892ce569be25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:56.952952 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af496a4a-0e95-4806-9fbd-892ce569be25 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:56.959980 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:56.973819 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 960/1920] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:25:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:25:57.127005 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df3716-a4c5-4934-94c8-11ad67c2e6cf None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:57.128879 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b2df3716-a4c5-4934-94c8-11ad67c2e6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:57.144449 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df3716-a4c5-4934-94c8-11ad67c2e6cf None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:57.151259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b992dfe-4c7f-4a8e-8e29-f2944945b1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:57.152350 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b992dfe-4c7f-4a8e-8e29-f2944945b1d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:57.152699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b992dfe-4c7f-4a8e-8e29-f2944945b1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:57.153058 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b992dfe-4c7f-4a8e-8e29-f2944945b1d2 admin admin] PATH_INFO: `/v3/users/fd0059d35acc476cb9962adc6ae8ec34` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:57.154808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b992dfe-4c7f-4a8e-8e29-f2944945b1d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd0059d35acc476cb9962adc6ae8ec34)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:57.157689 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b992dfe-4c7f-4a8e-8e29-f2944945b1d2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:57.252883 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4b992dfe-4c7f-4a8e-8e29-f2944945b1d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd0059d35acc476cb9962adc6ae8ec34'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:57.286455 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4b992dfe-4c7f-4a8e-8e29-f2944945b1d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd0059d35acc476cb9962adc6ae8ec34'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:57.324420 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4b992dfe-4c7f-4a8e-8e29-f2944945b1d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd0059d35acc476cb9962adc6ae8ec34'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:57.326921 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4b992dfe-4c7f-4a8e-8e29-f2944945b1d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd0059d35acc476cb9962adc6ae8ec34'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:57.328954 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4b992dfe-4c7f-4a8e-8e29-f2944945b1d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd0059d35acc476cb9962adc6ae8ec34'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:57.392965 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 961/1921] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:25:57 2022] DELETE /identity/v3/users/fd0059d35acc476cb9962adc6ae8ec34 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:25:57.401177 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96ad58-ef7a-485d-aef8-f48090a9f420 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:57.401706 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96ad58-ef7a-485d-aef8-f48090a9f420 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:57.402265 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96ad58-ef7a-485d-aef8-f48090a9f420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:57.431764 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7c96ad58-ef7a-485d-aef8-f48090a9f420 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:57.433007 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7c96ad58-ef7a-485d-aef8-f48090a9f420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:57.541188 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b992dfe-4c7f-4a8e-8e29-f2944945b1d2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:57.545705 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4b992dfe-4c7f-4a8e-8e29-f2944945b1d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:57.654515 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 961/1922] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:25:57.657212 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119cadf-6137-49f4-af6a-37530fc29991 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:57.657470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119cadf-6137-49f4-af6a-37530fc29991 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:57.658217 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119cadf-6137-49f4-af6a-37530fc29991 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:57.661566 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 962/1923] 10.222.0.18 () {54 vars in 921 bytes} [Mon Jul 25 19:25:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:25:57.664148 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a119cadf-6137-49f4-af6a-37530fc29991 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:57.669123 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a119cadf-6137-49f4-af6a-37530fc29991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:57.784145 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b992dfe-4c7f-4a8e-8e29-f2944945b1d2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:57.798434 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf222d8-dbbf-4441-8b27-d62afbd9ef5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:57.800466 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf222d8-dbbf-4441-8b27-d62afbd9ef5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:57.802581 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf222d8-dbbf-4441-8b27-d62afbd9ef5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:57.803212 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf222d8-dbbf-4441-8b27-d62afbd9ef5f admin admin] PATH_INFO: `/v3/projects/fc52b728c9bf46b998500900987f72a3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:57.806491 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf222d8-dbbf-4441-8b27-d62afbd9ef5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc52b728c9bf46b998500900987f72a3)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:57.808747 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a119cadf-6137-49f4-af6a-37530fc29991 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:57.810455 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf222d8-dbbf-4441-8b27-d62afbd9ef5f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:57.815032 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2681d6-2d19-4993-8dca-3f0f71b2d5dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:57.816342 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2681d6-2d19-4993-8dca-3f0f71b2d5dc service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:57.816800 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2681d6-2d19-4993-8dca-3f0f71b2d5dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:57.817239 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2681d6-2d19-4993-8dca-3f0f71b2d5dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:57.837446 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2681d6-2d19-4993-8dca-3f0f71b2d5dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:57.840260 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2681d6-2d19-4993-8dca-3f0f71b2d5dc service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:57.856101 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 963/1924] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:25:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:25:57.868639 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194b0c1-11d4-4124-90f4-cf622d282eb6 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:57.869492 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194b0c1-11d4-4124-90f4-cf622d282eb6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:57.869977 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194b0c1-11d4-4124-90f4-cf622d282eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:57.917074 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:57.918389 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4194b0c1-11d4-4124-90f4-cf622d282eb6 None None] MFA Rules not processed for user `860e7d26ad364795bb25836aadd4ac7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:57.923359 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4194b0c1-11d4-4124-90f4-cf622d282eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:57.936742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8bf222d8-dbbf-4441-8b27-d62afbd9ef5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc52b728c9bf46b998500900987f72a3', 'request_id': 'req-8bf222d8-dbbf-4441-8b27-d62afbd9ef5f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:57.962283 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8bf222d8-dbbf-4441-8b27-d62afbd9ef5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc52b728c9bf46b998500900987f72a3', 'request_id': 'req-8bf222d8-dbbf-4441-8b27-d62afbd9ef5f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:57.985336 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8bf222d8-dbbf-4441-8b27-d62afbd9ef5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc52b728c9bf46b998500900987f72a3', 'request_id': 'req-8bf222d8-dbbf-4441-8b27-d62afbd9ef5f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:58.020523 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8bf222d8-dbbf-4441-8b27-d62afbd9ef5f admin admin] The token cache is being invalidate because project fc52b728c9bf46b998500900987f72a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:25:58.021240 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8bf222d8-dbbf-4441-8b27-d62afbd9ef5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:58.022688 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8bf222d8-dbbf-4441-8b27-d62afbd9ef5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:58.031711 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 962/1925] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:25:57 2022] DELETE /identity/v3/projects/fc52b728c9bf46b998500900987f72a3 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:58.033420 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf222d8-dbbf-4441-8b27-d62afbd9ef5f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:58.038359 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8bf222d8-dbbf-4441-8b27-d62afbd9ef5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:58.156013 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 964/1926] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Jul 25 19:25:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:25:58.227947 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf222d8-dbbf-4441-8b27-d62afbd9ef5f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:58.248849 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413d57a-d548-4202-bade-8aeae49923f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:58.252158 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413d57a-d548-4202-bade-8aeae49923f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:58.252158 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413d57a-d548-4202-bade-8aeae49923f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:58.253689 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413d57a-d548-4202-bade-8aeae49923f4 admin admin] PATH_INFO: `/v3/users/7514086938a446cfb360af01aa24b24a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:58.258093 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8413d57a-d548-4202-bade-8aeae49923f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7514086938a446cfb360af01aa24b24a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:58.266183 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8413d57a-d548-4202-bade-8aeae49923f4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:58.381694 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8413d57a-d548-4202-bade-8aeae49923f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7514086938a446cfb360af01aa24b24a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:58.396818 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8413d57a-d548-4202-bade-8aeae49923f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7514086938a446cfb360af01aa24b24a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:58.413845 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8413d57a-d548-4202-bade-8aeae49923f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7514086938a446cfb360af01aa24b24a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:58.416420 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8413d57a-d548-4202-bade-8aeae49923f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7514086938a446cfb360af01aa24b24a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:58.417532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8413d57a-d548-4202-bade-8aeae49923f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7514086938a446cfb360af01aa24b24a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:58.458886 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 963/1927] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:25:58 2022] DELETE /identity/v3/users/7514086938a446cfb360af01aa24b24a => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:58.482554 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4194b0c1-11d4-4124-90f4-cf622d282eb6 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:58.493398 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4194b0c1-11d4-4124-90f4-cf622d282eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:58.513419 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4194b0c1-11d4-4124-90f4-cf622d282eb6 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:58.550455 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413d57a-d548-4202-bade-8aeae49923f4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:58.552923 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8413d57a-d548-4202-bade-8aeae49923f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:58.560697 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:58.677016 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:58.691937 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5711ee3-6ec6-4bc4-8218-014eee8f940b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:58.694573 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5711ee3-6ec6-4bc4-8218-014eee8f940b service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:58.695351 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5711ee3-6ec6-4bc4-8218-014eee8f940b service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:58.696685 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5711ee3-6ec6-4bc4-8218-014eee8f940b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:58.702740 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a5711ee3-6ec6-4bc4-8218-014eee8f940b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:58.725138 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413d57a-d548-4202-bade-8aeae49923f4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:58.735889 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33888f91-ac09-422e-b1f7-725cae49196c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:58.738765 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33888f91-ac09-422e-b1f7-725cae49196c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:58.739375 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33888f91-ac09-422e-b1f7-725cae49196c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:58.739949 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33888f91-ac09-422e-b1f7-725cae49196c admin admin] PATH_INFO: `/v3/projects/25243ec36d0f49f985fb689734d298df` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:58.741477 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5711ee3-6ec6-4bc4-8218-014eee8f940b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:58.745893 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33888f91-ac09-422e-b1f7-725cae49196c admin admin] RBAC: Authorizing `identity:delete_project(project_id=25243ec36d0f49f985fb689734d298df)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:58.746521 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5711ee3-6ec6-4bc4-8218-014eee8f940b service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:58.750237 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33888f91-ac09-422e-b1f7-725cae49196c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:58.773322 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 965/1928] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:25:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:25:58.893350 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-33888f91-ac09-422e-b1f7-725cae49196c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25243ec36d0f49f985fb689734d298df', 'request_id': 'req-33888f91-ac09-422e-b1f7-725cae49196c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:58.908211 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-33888f91-ac09-422e-b1f7-725cae49196c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25243ec36d0f49f985fb689734d298df', 'request_id': 'req-33888f91-ac09-422e-b1f7-725cae49196c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:58.922272 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-33888f91-ac09-422e-b1f7-725cae49196c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25243ec36d0f49f985fb689734d298df', 'request_id': 'req-33888f91-ac09-422e-b1f7-725cae49196c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:58.963958 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-33888f91-ac09-422e-b1f7-725cae49196c admin admin] The token cache is being invalidate because project 25243ec36d0f49f985fb689734d298df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:25:58.964834 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-33888f91-ac09-422e-b1f7-725cae49196c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:58.965811 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-33888f91-ac09-422e-b1f7-725cae49196c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:58.973645 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 964/1929] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:25:58 2022] DELETE /identity/v3/projects/25243ec36d0f49f985fb689734d298df => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:58.975774 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5711ee3-6ec6-4bc4-8218-014eee8f940b service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:58.995270 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5711ee3-6ec6-4bc4-8218-014eee8f940b service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:59.112487 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33888f91-ac09-422e-b1f7-725cae49196c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:59.114783 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-33888f91-ac09-422e-b1f7-725cae49196c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:59.127667 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4f0d11-09d8-48b4-882f-503c587b5ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:59.128911 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f0d11-09d8-48b4-882f-503c587b5ad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:59.130428 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f0d11-09d8-48b4-882f-503c587b5ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:59.130571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f0d11-09d8-48b4-882f-503c587b5ad6 admin admin] PATH_INFO: `/v3/users/7d112b8e69e24c1fa8a7dc6bd3d4dadf` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:59.132366 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4f0d11-09d8-48b4-882f-503c587b5ad6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d112b8e69e24c1fa8a7dc6bd3d4dadf)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:59.136330 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4f0d11-09d8-48b4-882f-503c587b5ad6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:59.138429 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33888f91-ac09-422e-b1f7-725cae49196c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:59.147984 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ddfca6-6423-418d-9cfd-c3520b11410e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:59.154129 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddfca6-6423-418d-9cfd-c3520b11410e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:59.154553 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddfca6-6423-418d-9cfd-c3520b11410e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:59.155022 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddfca6-6423-418d-9cfd-c3520b11410e admin admin] PATH_INFO: `/v3/projects/37d83134d1c4493a90241f74b348103d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:59.159121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ddfca6-6423-418d-9cfd-c3520b11410e admin admin] RBAC: Authorizing `identity:delete_project(project_id=37d83134d1c4493a90241f74b348103d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:59.167129 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ddfca6-6423-418d-9cfd-c3520b11410e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:59.243651 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-15ddfca6-6423-418d-9cfd-c3520b11410e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37d83134d1c4493a90241f74b348103d', 'request_id': 'req-15ddfca6-6423-418d-9cfd-c3520b11410e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:59.253365 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-15ddfca6-6423-418d-9cfd-c3520b11410e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37d83134d1c4493a90241f74b348103d', 'request_id': 'req-15ddfca6-6423-418d-9cfd-c3520b11410e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:59.267500 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1b4f0d11-09d8-48b4-882f-503c587b5ad6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d112b8e69e24c1fa8a7dc6bd3d4dadf'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:59.274905 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-15ddfca6-6423-418d-9cfd-c3520b11410e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37d83134d1c4493a90241f74b348103d', 'request_id': 'req-15ddfca6-6423-418d-9cfd-c3520b11410e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:59.283497 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1b4f0d11-09d8-48b4-882f-503c587b5ad6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d112b8e69e24c1fa8a7dc6bd3d4dadf'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:59.298968 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-15ddfca6-6423-418d-9cfd-c3520b11410e admin admin] The token cache is being invalidate because project 37d83134d1c4493a90241f74b348103d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:25:59.299656 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-15ddfca6-6423-418d-9cfd-c3520b11410e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:59.299796 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1b4f0d11-09d8-48b4-882f-503c587b5ad6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d112b8e69e24c1fa8a7dc6bd3d4dadf'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:59.300936 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-15ddfca6-6423-418d-9cfd-c3520b11410e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:59.301093 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1b4f0d11-09d8-48b4-882f-503c587b5ad6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d112b8e69e24c1fa8a7dc6bd3d4dadf'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:59.303952 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 965/1930] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:25:59 2022] DELETE /identity/v3/projects/37d83134d1c4493a90241f74b348103d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:59.313158 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe1d0e-3741-4aa0-bc28-50e4e93251cd None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:59.313665 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe1d0e-3741-4aa0-bc28-50e4e93251cd None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:59.314152 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe1d0e-3741-4aa0-bc28-50e4e93251cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:59.325535 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1b4f0d11-09d8-48b4-882f-503c587b5ad6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d112b8e69e24c1fa8a7dc6bd3d4dadf'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:25:59.333898 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 966/1931] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:25:58 2022] DELETE /identity/v3/users/7d112b8e69e24c1fa8a7dc6bd3d4dadf => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:25:59.349435 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4dfe1d0e-3741-4aa0-bc28-50e4e93251cd None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:59.351758 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4dfe1d0e-3741-4aa0-bc28-50e4e93251cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:59.463962 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 966/1932] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:25:59.478831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736151c3-4f72-4e26-ad81-d050ccc79bed None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:59.479158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736151c3-4f72-4e26-ad81-d050ccc79bed None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:59.479744 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736151c3-4f72-4e26-ad81-d050ccc79bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:59.516502 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-736151c3-4f72-4e26-ad81-d050ccc79bed None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:25:59.518772 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-736151c3-4f72-4e26-ad81-d050ccc79bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:25:59.537792 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 967/1933] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:25:59.545721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfe1d0e-3741-4aa0-bc28-50e4e93251cd None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:59.564701 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:59.578396 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfe1d0e-3741-4aa0-bc28-50e4e93251cd None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:59.593427 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efadec9b-c516-4ed2-86c6-537bdcfa2996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:59.594219 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efadec9b-c516-4ed2-86c6-537bdcfa2996 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:59.596333 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efadec9b-c516-4ed2-86c6-537bdcfa2996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:59.596759 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efadec9b-c516-4ed2-86c6-537bdcfa2996 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:59.597666 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efadec9b-c516-4ed2-86c6-537bdcfa2996 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:25:59.598346 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efadec9b-c516-4ed2-86c6-537bdcfa2996 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:59.600206 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efadec9b-c516-4ed2-86c6-537bdcfa2996 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:59.629082 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 967/1934] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:25:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:25:59.631249 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736151c3-4f72-4e26-ad81-d050ccc79bed None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:59.648288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736151c3-4f72-4e26-ad81-d050ccc79bed None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:59.655624 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2098c10-6ee6-4724-aed4-c5e9fe574c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:59.656576 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2098c10-6ee6-4724-aed4-c5e9fe574c05 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:59.656971 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2098c10-6ee6-4724-aed4-c5e9fe574c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:59.657506 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2098c10-6ee6-4724-aed4-c5e9fe574c05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:59.658299 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2098c10-6ee6-4724-aed4-c5e9fe574c05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:59.661124 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2098c10-6ee6-4724-aed4-c5e9fe574c05 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:59.704469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 968/1935] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:25:59 2022] POST /identity/v3/projects => generated 385 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:25:59.709172 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efadec9b-c516-4ed2-86c6-537bdcfa2996 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:59.724283 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efadec9b-c516-4ed2-86c6-537bdcfa2996 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:59.846493 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712b4120-6ed8-4430-997c-98d19356ce49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:59.847741 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712b4120-6ed8-4430-997c-98d19356ce49 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:59.848247 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712b4120-6ed8-4430-997c-98d19356ce49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:59.848753 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712b4120-6ed8-4430-997c-98d19356ce49 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:59.850168 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712b4120-6ed8-4430-997c-98d19356ce49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:59.854883 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712b4120-6ed8-4430-997c-98d19356ce49 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:59.904634 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 968/1936] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:25:59 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:25:59.906831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2098c10-6ee6-4724-aed4-c5e9fe574c05 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:25:59.927238 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2098c10-6ee6-4724-aed4-c5e9fe574c05 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:25:59.944022 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad24d76-c8d6-4e42-83b8-85ca83ecc455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:25:59.945239 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad24d76-c8d6-4e42-83b8-85ca83ecc455 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:25:59.945775 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad24d76-c8d6-4e42-83b8-85ca83ecc455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:25:59.946749 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad24d76-c8d6-4e42-83b8-85ca83ecc455 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:25:59.947683 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad24d76-c8d6-4e42-83b8-85ca83ecc455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:25:59.950979 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad24d76-c8d6-4e42-83b8-85ca83ecc455 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:25:59.955684 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:25:59.975310 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 969/1937] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:25:59.983981 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712b4120-6ed8-4430-997c-98d19356ce49 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:00.005865 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712b4120-6ed8-4430-997c-98d19356ce49 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:00.018673 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ee3e0-0cd9-4b92-869b-c9cc0b86bfd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:00.020192 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ee3e0-0cd9-4b92-869b-c9cc0b86bfd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:00.020911 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ee3e0-0cd9-4b92-869b-c9cc0b86bfd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:00.021794 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ee3e0-0cd9-4b92-869b-c9cc0b86bfd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:00.023554 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ee3e0-0cd9-4b92-869b-c9cc0b86bfd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:00.027468 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ee3e0-0cd9-4b92-869b-c9cc0b86bfd7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:00.061854 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 969/1938] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:26:00.063953 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad24d76-c8d6-4e42-83b8-85ca83ecc455 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:00.088059 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad24d76-c8d6-4e42-83b8-85ca83ecc455 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:00.097135 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5930c57-d5c7-4211-be0d-e6188b0c33d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:00.098801 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5930c57-d5c7-4211-be0d-e6188b0c33d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:00.099579 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5930c57-d5c7-4211-be0d-e6188b0c33d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:00.100091 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5930c57-d5c7-4211-be0d-e6188b0c33d7 admin admin] PATH_INFO: `/v3/projects/c22f95fad9604219b80c1899bdb396d0/users/478ba48725fb421784bd549f149ce3fc/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:00.137852 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5930c57-d5c7-4211-be0d-e6188b0c33d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c22f95fad9604219b80c1899bdb396d0, user_id=478ba48725fb421784bd549f149ce3fc, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:00.143715 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5930c57-d5c7-4211-be0d-e6188b0c33d7 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:00.175582 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 970/1939] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:26:00 2022] PUT /identity/v3/projects/c22f95fad9604219b80c1899bdb396d0/users/478ba48725fb421784bd549f149ce3fc/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:26:00.194245 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3def38d-0798-49cd-9f8b-7c8f041933bc None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:00.194725 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3def38d-0798-49cd-9f8b-7c8f041933bc None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:00.195249 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3def38d-0798-49cd-9f8b-7c8f041933bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:00.205032 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5930c57-d5c7-4211-be0d-e6188b0c33d7 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:00.208921 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b5930c57-d5c7-4211-be0d-e6188b0c33d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:00.270718 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b3def38d-0798-49cd-9f8b-7c8f041933bc None None] MFA Rules not processed for user `478ba48725fb421784bd549f149ce3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:00.275418 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b3def38d-0798-49cd-9f8b-7c8f041933bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:00.429935 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5930c57-d5c7-4211-be0d-e6188b0c33d7 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:00.439855 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d1b5d9-eba9-47a5-a9ed-cbf83c3bee5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:00.441464 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d1b5d9-eba9-47a5-a9ed-cbf83c3bee5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:00.442021 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d1b5d9-eba9-47a5-a9ed-cbf83c3bee5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:00.442470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d1b5d9-eba9-47a5-a9ed-cbf83c3bee5f admin admin] PATH_INFO: `/v3/projects/22a61edbd6f34cb7935555d9b0e45367` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:00.445082 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d1b5d9-eba9-47a5-a9ed-cbf83c3bee5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=22a61edbd6f34cb7935555d9b0e45367)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:00.449689 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d1b5d9-eba9-47a5-a9ed-cbf83c3bee5f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:00.460679 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 970/1940] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:26:00.500195 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3def38d-0798-49cd-9f8b-7c8f041933bc None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:00.504761 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b3def38d-0798-49cd-9f8b-7c8f041933bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:00.572105 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-50d1b5d9-eba9-47a5-a9ed-cbf83c3bee5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22a61edbd6f34cb7935555d9b0e45367', 'request_id': 'req-50d1b5d9-eba9-47a5-a9ed-cbf83c3bee5f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:00.587053 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-50d1b5d9-eba9-47a5-a9ed-cbf83c3bee5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22a61edbd6f34cb7935555d9b0e45367', 'request_id': 'req-50d1b5d9-eba9-47a5-a9ed-cbf83c3bee5f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:00.598393 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-50d1b5d9-eba9-47a5-a9ed-cbf83c3bee5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22a61edbd6f34cb7935555d9b0e45367', 'request_id': 'req-50d1b5d9-eba9-47a5-a9ed-cbf83c3bee5f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:00.620992 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-50d1b5d9-eba9-47a5-a9ed-cbf83c3bee5f admin admin] The token cache is being invalidate because project 22a61edbd6f34cb7935555d9b0e45367 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:26:00.621449 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-50d1b5d9-eba9-47a5-a9ed-cbf83c3bee5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:00.622699 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-50d1b5d9-eba9-47a5-a9ed-cbf83c3bee5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:00.626195 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 971/1941] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:26:00 2022] DELETE /identity/v3/projects/22a61edbd6f34cb7935555d9b0e45367 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:26:00.637216 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b0d6e-39fa-4320-b45f-27c4f66eaf50 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:00.637659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b0d6e-39fa-4320-b45f-27c4f66eaf50 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:00.639221 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b0d6e-39fa-4320-b45f-27c4f66eaf50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:00.643827 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3def38d-0798-49cd-9f8b-7c8f041933bc None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:00.670531 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:00.678804 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4d5b0d6e-39fa-4320-b45f-27c4f66eaf50 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:00.682787 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4d5b0d6e-39fa-4320-b45f-27c4f66eaf50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:00.756365 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7debdda8-9764-4da9-bfed-79ce678a0aa7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:00.756761 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7debdda8-9764-4da9-bfed-79ce678a0aa7 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:00.757183 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7debdda8-9764-4da9-bfed-79ce678a0aa7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:00.757643 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7debdda8-9764-4da9-bfed-79ce678a0aa7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:00.761499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7debdda8-9764-4da9-bfed-79ce678a0aa7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:00.862414 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 972/1942] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:26:00.877065 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d66ad-35db-4fdc-8c33-1899d9b116e3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:00.877591 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d66ad-35db-4fdc-8c33-1899d9b116e3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:00.878098 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d66ad-35db-4fdc-8c33-1899d9b116e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:00.896856 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:00.920725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f89d66ad-35db-4fdc-8c33-1899d9b116e3 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:00.923972 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f89d66ad-35db-4fdc-8c33-1899d9b116e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:00.932073 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7debdda8-9764-4da9-bfed-79ce678a0aa7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:00.934344 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7debdda8-9764-4da9-bfed-79ce678a0aa7 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:00.943453 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 973/1943] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:26:00.950239 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 971/1944] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:26:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:26:00.955044 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d66ad-35db-4fdc-8c33-1899d9b116e3 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:00.976057 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d66ad-35db-4fdc-8c33-1899d9b116e3 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:00.981220 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7debdda8-9764-4da9-bfed-79ce678a0aa7 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:00.983817 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e8610-c3bc-465d-952d-b6c5fa7b1c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:00.984648 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7debdda8-9764-4da9-bfed-79ce678a0aa7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:00.985091 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e8610-c3bc-465d-952d-b6c5fa7b1c7e admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:00.985555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e8610-c3bc-465d-952d-b6c5fa7b1c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:00.986052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e8610-c3bc-465d-952d-b6c5fa7b1c7e admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:00.987233 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e8610-c3bc-465d-952d-b6c5fa7b1c7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:26:00.987785 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e8610-c3bc-465d-952d-b6c5fa7b1c7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:00.990304 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e8610-c3bc-465d-952d-b6c5fa7b1c7e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:01.013947 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 974/1945] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:26:01.019131 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e8610-c3bc-465d-952d-b6c5fa7b1c7e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:01.019912 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7debdda8-9764-4da9-bfed-79ce678a0aa7 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:01.033717 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac70794-5f17-4492-95b9-98aca56fd3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:01.034714 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac70794-5f17-4492-95b9-98aca56fd3a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:01.035062 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac70794-5f17-4492-95b9-98aca56fd3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:01.035351 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac70794-5f17-4492-95b9-98aca56fd3a5 admin admin] PATH_INFO: `/v3/users/478ba48725fb421784bd549f149ce3fc` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:01.036875 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac70794-5f17-4492-95b9-98aca56fd3a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=478ba48725fb421784bd549f149ce3fc)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:01.039070 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac70794-5f17-4492-95b9-98aca56fd3a5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:01.041461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e8610-c3bc-465d-952d-b6c5fa7b1c7e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:01.055190 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9329723-375d-4884-b391-dd47a8c74074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:01.056878 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9329723-375d-4884-b391-dd47a8c74074 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:01.059052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9329723-375d-4884-b391-dd47a8c74074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:01.059685 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9329723-375d-4884-b391-dd47a8c74074 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:01.062316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9329723-375d-4884-b391-dd47a8c74074 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:01.066389 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9329723-375d-4884-b391-dd47a8c74074 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:01.149930 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 975/1946] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:26:01 2022] POST /identity/v3/projects => generated 381 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:26:01.158234 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9329723-375d-4884-b391-dd47a8c74074 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:01.188404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bac70794-5f17-4492-95b9-98aca56fd3a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '478ba48725fb421784bd549f149ce3fc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:01.210525 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bac70794-5f17-4492-95b9-98aca56fd3a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '478ba48725fb421784bd549f149ce3fc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:01.232944 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bac70794-5f17-4492-95b9-98aca56fd3a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '478ba48725fb421784bd549f149ce3fc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:01.236897 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bac70794-5f17-4492-95b9-98aca56fd3a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '478ba48725fb421784bd549f149ce3fc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:01.238033 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bac70794-5f17-4492-95b9-98aca56fd3a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '478ba48725fb421784bd549f149ce3fc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:01.309346 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 972/1947] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:26:00 2022] DELETE /identity/v3/users/478ba48725fb421784bd549f149ce3fc => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:26:01.313037 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f014351-70fd-40cb-8d9f-0ebbc6ec2c9d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:01.313494 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f014351-70fd-40cb-8d9f-0ebbc6ec2c9d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:01.314150 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f014351-70fd-40cb-8d9f-0ebbc6ec2c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:01.371368 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9329723-375d-4884-b391-dd47a8c74074 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:01.409011 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5f014351-70fd-40cb-8d9f-0ebbc6ec2c9d None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:01.411780 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5f014351-70fd-40cb-8d9f-0ebbc6ec2c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:01.532214 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:01.584967 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 973/1948] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:26:01.607131 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f014351-70fd-40cb-8d9f-0ebbc6ec2c9d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:01.610429 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5f014351-70fd-40cb-8d9f-0ebbc6ec2c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:01.657255 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474125d2-1979-4256-8862-715998cca1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:01.660445 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474125d2-1979-4256-8862-715998cca1ea admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:01.663789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474125d2-1979-4256-8862-715998cca1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:01.665431 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474125d2-1979-4256-8862-715998cca1ea admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:01.667752 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474125d2-1979-4256-8862-715998cca1ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:01.674784 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474125d2-1979-4256-8862-715998cca1ea admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:01.740283 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 976/1949] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:26:01 2022] POST /identity/v3/users => generated 291 bytes in 576 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:26:01.747639 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474125d2-1979-4256-8862-715998cca1ea admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:01.756949 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-474125d2-1979-4256-8862-715998cca1ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:01.794286 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474125d2-1979-4256-8862-715998cca1ea admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:01.801447 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4a7a9-397b-46dc-bd72-7e61ed15396b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:01.802703 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4a7a9-397b-46dc-bd72-7e61ed15396b admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:01.803742 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4a7a9-397b-46dc-bd72-7e61ed15396b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:01.804342 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4a7a9-397b-46dc-bd72-7e61ed15396b admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:01.806449 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f4a7a9-397b-46dc-bd72-7e61ed15396b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:01.809835 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f4a7a9-397b-46dc-bd72-7e61ed15396b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:01.827980 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f014351-70fd-40cb-8d9f-0ebbc6ec2c9d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:01.844097 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 977/1950] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:26:01.846103 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4a7a9-397b-46dc-bd72-7e61ed15396b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:01.846713 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facec4ac-fadc-4780-8800-4d70eef96e59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:01.848186 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facec4ac-fadc-4780-8800-4d70eef96e59 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:01.848440 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facec4ac-fadc-4780-8800-4d70eef96e59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:01.848890 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facec4ac-fadc-4780-8800-4d70eef96e59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:01.869360 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4a7a9-397b-46dc-bd72-7e61ed15396b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:01.876715 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e9ad6-210e-4b4b-9ac4-ddd535044f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:01.879381 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e9ad6-210e-4b4b-9ac4-ddd535044f3d admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:01.880249 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e9ad6-210e-4b4b-9ac4-ddd535044f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:01.881013 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e9ad6-210e-4b4b-9ac4-ddd535044f3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:01.885273 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916e9ad6-210e-4b4b-9ac4-ddd535044f3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:01.888664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916e9ad6-210e-4b4b-9ac4-ddd535044f3d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:01.894715 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facec4ac-fadc-4780-8800-4d70eef96e59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:01.896753 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facec4ac-fadc-4780-8800-4d70eef96e59 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:01.917104 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 978/1951] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:26:01.919091 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e9ad6-210e-4b4b-9ac4-ddd535044f3d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:01.930582 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 974/1952] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:26:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:26:01.954353 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e9ad6-210e-4b4b-9ac4-ddd535044f3d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:01.978153 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792bd78a-6ae9-46e9-96d9-72087501a6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:01.982077 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792bd78a-6ae9-46e9-96d9-72087501a6e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:01.983027 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792bd78a-6ae9-46e9-96d9-72087501a6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:01.988659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792bd78a-6ae9-46e9-96d9-72087501a6e7 admin admin] PATH_INFO: `/v3/projects/36eea90f360646829dad58a5e15e9338/users/5ca77871109e4e1da54b8b7e0a3fd049/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:02.049824 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792bd78a-6ae9-46e9-96d9-72087501a6e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36eea90f360646829dad58a5e15e9338, user_id=5ca77871109e4e1da54b8b7e0a3fd049, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:02.055764 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792bd78a-6ae9-46e9-96d9-72087501a6e7 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:02.097779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 979/1953] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:26:01 2022] PUT /identity/v3/projects/36eea90f360646829dad58a5e15e9338/users/5ca77871109e4e1da54b8b7e0a3fd049/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:26:02.119164 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c46f6-4849-4312-b341-3dd875e6e10c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:02.119572 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c46f6-4849-4312-b341-3dd875e6e10c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:02.120855 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c46f6-4849-4312-b341-3dd875e6e10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:02.299800 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-513c46f6-4849-4312-b341-3dd875e6e10c None None] MFA Rules not processed for user `5ca77871109e4e1da54b8b7e0a3fd049`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:02.302431 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-513c46f6-4849-4312-b341-3dd875e6e10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:02.595457 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 975/1954] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 480 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:26:02.610070 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792bd78a-6ae9-46e9-96d9-72087501a6e7 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:02.610283 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c46f6-4849-4312-b341-3dd875e6e10c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:02.614662 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-513c46f6-4849-4312-b341-3dd875e6e10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:02.666370 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:02.843582 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c46f6-4849-4312-b341-3dd875e6e10c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:02.855456 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130cc552-0f07-4daf-abf5-f63dbc29623b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:02.855456 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130cc552-0f07-4daf-abf5-f63dbc29623b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:02.855456 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130cc552-0f07-4daf-abf5-f63dbc29623b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:02.855456 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130cc552-0f07-4daf-abf5-f63dbc29623b admin admin] PATH_INFO: `/v3/users/938ac4f195cd466199de039650614012` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:02.863658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130cc552-0f07-4daf-abf5-f63dbc29623b admin admin] RBAC: Authorizing `identity:delete_user(user_id=938ac4f195cd466199de039650614012)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:02.877748 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130cc552-0f07-4daf-abf5-f63dbc29623b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:02.891563 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792bd78a-6ae9-46e9-96d9-72087501a6e7 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:02.902207 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275e567-6fcc-49d8-9059-63d57d22f678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:02.904029 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275e567-6fcc-49d8-9059-63d57d22f678 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:02.904352 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275e567-6fcc-49d8-9059-63d57d22f678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:02.904822 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275e567-6fcc-49d8-9059-63d57d22f678 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:02.904977 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:02.905631 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7275e567-6fcc-49d8-9059-63d57d22f678 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:02.907724 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7275e567-6fcc-49d8-9059-63d57d22f678 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:03.014269 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 980/1955] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:26:02 2022] POST /identity/v3/projects => generated 383 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:26:03.014269 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275e567-6fcc-49d8-9059-63d57d22f678 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:03.016057 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-130cc552-0f07-4daf-abf5-f63dbc29623b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '938ac4f195cd466199de039650614012'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:03.017704 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7275e567-6fcc-49d8-9059-63d57d22f678 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:03.033669 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-130cc552-0f07-4daf-abf5-f63dbc29623b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '938ac4f195cd466199de039650614012'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:03.054353 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-130cc552-0f07-4daf-abf5-f63dbc29623b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '938ac4f195cd466199de039650614012'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:03.056926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-130cc552-0f07-4daf-abf5-f63dbc29623b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '938ac4f195cd466199de039650614012'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:03.060367 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-130cc552-0f07-4daf-abf5-f63dbc29623b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '938ac4f195cd466199de039650614012'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:03.106250 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 976/1956] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:26:02 2022] DELETE /identity/v3/users/938ac4f195cd466199de039650614012 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:26:03.113468 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130cc552-0f07-4daf-abf5-f63dbc29623b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:03.119058 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-130cc552-0f07-4daf-abf5-f63dbc29623b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:03.238434 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275e567-6fcc-49d8-9059-63d57d22f678 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:03.332557 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130cc552-0f07-4daf-abf5-f63dbc29623b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:03.347099 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eedb49-d473-4b4c-9371-f1d2941ed03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:03.348957 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eedb49-d473-4b4c-9371-f1d2941ed03d admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:03.349201 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eedb49-d473-4b4c-9371-f1d2941ed03d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:03.349659 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eedb49-d473-4b4c-9371-f1d2941ed03d admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:03.350551 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eedb49-d473-4b4c-9371-f1d2941ed03d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:03.354405 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eedb49-d473-4b4c-9371-f1d2941ed03d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:03.407681 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 977/1957] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:26:03 2022] POST /identity/v3/users => generated 298 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:26:03.415268 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb1676-1539-4cf0-96ce-a71e869d690a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:03.415268 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb1676-1539-4cf0-96ce-a71e869d690a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:03.415268 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb1676-1539-4cf0-96ce-a71e869d690a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:03.435868 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f6110-5996-4448-aae3-0fec7b6737a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:03.443873 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f6110-5996-4448-aae3-0fec7b6737a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:03.446552 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f6110-5996-4448-aae3-0fec7b6737a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:03.448113 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f6110-5996-4448-aae3-0fec7b6737a3 admin admin] PATH_INFO: `/v3/projects/c22f95fad9604219b80c1899bdb396d0` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:03.450710 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-aedb1676-1539-4cf0-96ce-a71e869d690a None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:03.452987 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-aedb1676-1539-4cf0-96ce-a71e869d690a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:03.455875 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f6110-5996-4448-aae3-0fec7b6737a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c22f95fad9604219b80c1899bdb396d0)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:03.465407 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 978/1958] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:26:03.466238 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb1676-1539-4cf0-96ce-a71e869d690a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:03.501340 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f6110-5996-4448-aae3-0fec7b6737a3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:03.520257 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb1676-1539-4cf0-96ce-a71e869d690a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:03.532157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f76dda-ed67-4888-aa8e-000e04261d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:03.533105 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f76dda-ed67-4888-aa8e-000e04261d43 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:03.533348 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f76dda-ed67-4888-aa8e-000e04261d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:03.533702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f76dda-ed67-4888-aa8e-000e04261d43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:03.535629 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f76dda-ed67-4888-aa8e-000e04261d43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:03.537317 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f76dda-ed67-4888-aa8e-000e04261d43 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:03.554964 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 979/1959] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:26:03.556854 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f76dda-ed67-4888-aa8e-000e04261d43 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:03.563812 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c3f76dda-ed67-4888-aa8e-000e04261d43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:03.677999 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:03.691821 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4b1f6110-5996-4448-aae3-0fec7b6737a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c22f95fad9604219b80c1899bdb396d0', 'request_id': 'req-4b1f6110-5996-4448-aae3-0fec7b6737a3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:03.715711 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4b1f6110-5996-4448-aae3-0fec7b6737a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c22f95fad9604219b80c1899bdb396d0', 'request_id': 'req-4b1f6110-5996-4448-aae3-0fec7b6737a3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:03.722985 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f76dda-ed67-4888-aa8e-000e04261d43 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:03.737648 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4b1f6110-5996-4448-aae3-0fec7b6737a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c22f95fad9604219b80c1899bdb396d0', 'request_id': 'req-4b1f6110-5996-4448-aae3-0fec7b6737a3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:03.800877 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4b1f6110-5996-4448-aae3-0fec7b6737a3 admin admin] The token cache is being invalidate because project c22f95fad9604219b80c1899bdb396d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:26:03.801426 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4b1f6110-5996-4448-aae3-0fec7b6737a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:03.810641 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4b1f6110-5996-4448-aae3-0fec7b6737a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:03.815765 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 981/1960] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:26:03 2022] DELETE /identity/v3/projects/c22f95fad9604219b80c1899bdb396d0 => generated 0 bytes in 806 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:26:03.817336 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f6110-5996-4448-aae3-0fec7b6737a3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:03.822976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4b1f6110-5996-4448-aae3-0fec7b6737a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:03.844544 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:03.928848 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ed3d3-6ca5-4c98-bf4b-1c6a3fe1cbfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:03.930111 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ed3d3-6ca5-4c98-bf4b-1c6a3fe1cbfc service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:03.931313 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ed3d3-6ca5-4c98-bf4b-1c6a3fe1cbfc service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:03.931313 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ed3d3-6ca5-4c98-bf4b-1c6a3fe1cbfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:03.934863 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-100ed3d3-6ca5-4c98-bf4b-1c6a3fe1cbfc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:04.025423 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f6110-5996-4448-aae3-0fec7b6737a3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:04.047646 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd5f197-ad6a-45c1-8f8d-800de90dd56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:04.053845 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5f197-ad6a-45c1-8f8d-800de90dd56c admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:04.054449 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5f197-ad6a-45c1-8f8d-800de90dd56c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:04.055121 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5f197-ad6a-45c1-8f8d-800de90dd56c admin admin] PATH_INFO: `/v3/projects/f93ae75cfc5f48b383cea98cf7f8fcc0/users/db8808aacd7b4f789bc59944e9ff0499/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:04.097437 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd5f197-ad6a-45c1-8f8d-800de90dd56c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f93ae75cfc5f48b383cea98cf7f8fcc0, user_id=db8808aacd7b4f789bc59944e9ff0499, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:04.106633 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd5f197-ad6a-45c1-8f8d-800de90dd56c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:04.131725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 982/1961] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:26:03 2022] PUT /identity/v3/projects/f93ae75cfc5f48b383cea98cf7f8fcc0/users/db8808aacd7b4f789bc59944e9ff0499/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:26:04.134770 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bab5b-83c4-4741-b871-ae04fc55173f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:04.136115 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bab5b-83c4-4741-b871-ae04fc55173f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:04.137049 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bab5b-83c4-4741-b871-ae04fc55173f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:04.148116 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100ed3d3-6ca5-4c98-bf4b-1c6a3fe1cbfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:04.149365 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100ed3d3-6ca5-4c98-bf4b-1c6a3fe1cbfc service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:04.175251 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 980/1962] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:26:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:26:04.176867 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ed3d3-6ca5-4c98-bf4b-1c6a3fe1cbfc service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:04.206481 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ed3d3-6ca5-4c98-bf4b-1c6a3fe1cbfc service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:04.229419 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-799bab5b-83c4-4741-b871-ae04fc55173f None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:04.231582 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-799bab5b-83c4-4741-b871-ae04fc55173f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:04.476362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 983/1963] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:26:04.488782 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f081b-9171-45e2-a641-31c594b28ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:04.489262 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b61208-715b-4de5-8987-ef08cdd7edcd None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:04.490074 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b61208-715b-4de5-8987-ef08cdd7edcd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:04.491205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b61208-715b-4de5-8987-ef08cdd7edcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:04.493186 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f081b-9171-45e2-a641-31c594b28ef6 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:04.493969 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f081b-9171-45e2-a641-31c594b28ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:04.494897 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f081b-9171-45e2-a641-31c594b28ef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:04.496613 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f081b-9171-45e2-a641-31c594b28ef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:04.501012 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f081b-9171-45e2-a641-31c594b28ef6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:04.523414 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 981/1964] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:26:04 2022] GET /identity/v3/roles => generated 1420 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:26:04.533874 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f081b-9171-45e2-a641-31c594b28ef6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:04.556313 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f081b-9171-45e2-a641-31c594b28ef6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:04.559368 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-97b61208-715b-4de5-8987-ef08cdd7edcd None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:04.562142 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-97b61208-715b-4de5-8987-ef08cdd7edcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:04.568840 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a649ed-ad29-445d-a0e7-3a10a98d98f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:04.570310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a649ed-ad29-445d-a0e7-3a10a98d98f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:04.571909 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a649ed-ad29-445d-a0e7-3a10a98d98f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:04.572596 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a649ed-ad29-445d-a0e7-3a10a98d98f0 admin admin] PATH_INFO: `/v3/projects/f93ae75cfc5f48b383cea98cf7f8fcc0/users/db8808aacd7b4f789bc59944e9ff0499/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:04.578116 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a649ed-ad29-445d-a0e7-3a10a98d98f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f93ae75cfc5f48b383cea98cf7f8fcc0, user_id=db8808aacd7b4f789bc59944e9ff0499, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:04.584842 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a649ed-ad29-445d-a0e7-3a10a98d98f0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:04.605177 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 984/1965] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:26:04.606897 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b61208-715b-4de5-8987-ef08cdd7edcd None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:04.610739 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 982/1966] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:26:04 2022] PUT /identity/v3/projects/f93ae75cfc5f48b383cea98cf7f8fcc0/users/db8808aacd7b4f789bc59944e9ff0499/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:26:04.629922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf995c-0465-4d04-abf5-aa668c29b500 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:04.634044 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf995c-0465-4d04-abf5-aa668c29b500 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:04.635245 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf995c-0465-4d04-abf5-aa668c29b500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:04.647950 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b61208-715b-4de5-8987-ef08cdd7edcd None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:04.741424 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-abbf995c-0465-4d04-abf5-aa668c29b500 None None] MFA Rules not processed for user `db8808aacd7b4f789bc59944e9ff0499`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:04.741424 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-abbf995c-0465-4d04-abf5-aa668c29b500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:04.782824 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:04.819089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b255e86-f8bc-4cf2-95a2-6dba007b182e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:04.821022 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b255e86-f8bc-4cf2-95a2-6dba007b182e admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:04.821527 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b255e86-f8bc-4cf2-95a2-6dba007b182e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:04.822024 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b255e86-f8bc-4cf2-95a2-6dba007b182e admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:04.823466 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b255e86-f8bc-4cf2-95a2-6dba007b182e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:26:04.824053 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b255e86-f8bc-4cf2-95a2-6dba007b182e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:04.827920 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b255e86-f8bc-4cf2-95a2-6dba007b182e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:04.853068 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:04.863659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 985/1967] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:26:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:26:04.873945 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b255e86-f8bc-4cf2-95a2-6dba007b182e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:04.899049 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b255e86-f8bc-4cf2-95a2-6dba007b182e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:04.911666 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69316a20-4f94-4f5f-970b-fbbac7a009cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:04.913262 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69316a20-4f94-4f5f-970b-fbbac7a009cf admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:04.913976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69316a20-4f94-4f5f-970b-fbbac7a009cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:04.915585 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69316a20-4f94-4f5f-970b-fbbac7a009cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:04.918469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69316a20-4f94-4f5f-970b-fbbac7a009cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:04.922998 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69316a20-4f94-4f5f-970b-fbbac7a009cf admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:04.947572 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 983/1968] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:26:04.960159 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf995c-0465-4d04-abf5-aa668c29b500 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:04.965788 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-abbf995c-0465-4d04-abf5-aa668c29b500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:04.999806 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 986/1969] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:26:04 2022] POST /identity/v3/projects => generated 369 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:26:05.003931 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69316a20-4f94-4f5f-970b-fbbac7a009cf admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:05.138540 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69316a20-4f94-4f5f-970b-fbbac7a009cf admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:05.143565 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf995c-0465-4d04-abf5-aa668c29b500 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:05.143982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f28d1-e4e5-44ed-bd40-cc7ef249dda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:05.145368 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f28d1-e4e5-44ed-bd40-cc7ef249dda3 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:05.145368 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f28d1-e4e5-44ed-bd40-cc7ef249dda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:05.145368 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f28d1-e4e5-44ed-bd40-cc7ef249dda3 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:05.146897 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f28d1-e4e5-44ed-bd40-cc7ef249dda3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:05.150109 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739f28d1-e4e5-44ed-bd40-cc7ef249dda3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:05.204493 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 987/1970] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:26:05 2022] POST /identity/v3/users => generated 285 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:26:05.210213 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f28d1-e4e5-44ed-bd40-cc7ef249dda3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:05.244700 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f28d1-e4e5-44ed-bd40-cc7ef249dda3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:05.252527 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49724cf0-68a1-4df7-9fc5-3e5f9ae6fb22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:05.255821 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49724cf0-68a1-4df7-9fc5-3e5f9ae6fb22 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:05.255821 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49724cf0-68a1-4df7-9fc5-3e5f9ae6fb22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:05.255821 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49724cf0-68a1-4df7-9fc5-3e5f9ae6fb22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:05.256150 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49724cf0-68a1-4df7-9fc5-3e5f9ae6fb22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:05.272564 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49724cf0-68a1-4df7-9fc5-3e5f9ae6fb22 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:05.295740 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 988/1971] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:26:05.296206 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734077f6-425e-4c0f-94e4-c3bc68e1533d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:05.296396 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49724cf0-68a1-4df7-9fc5-3e5f9ae6fb22 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:05.296523 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734077f6-425e-4c0f-94e4-c3bc68e1533d service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:05.296523 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734077f6-425e-4c0f-94e4-c3bc68e1533d service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:05.296523 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734077f6-425e-4c0f-94e4-c3bc68e1533d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:05.300178 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-49724cf0-68a1-4df7-9fc5-3e5f9ae6fb22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:05.316211 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49724cf0-68a1-4df7-9fc5-3e5f9ae6fb22 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:05.325519 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0172ab2e-04c8-4f00-8cf0-43f1bd090f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:05.326436 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0172ab2e-04c8-4f00-8cf0-43f1bd090f16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:05.326726 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0172ab2e-04c8-4f00-8cf0-43f1bd090f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:05.327008 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0172ab2e-04c8-4f00-8cf0-43f1bd090f16 admin admin] PATH_INFO: `/v3/users/c1feeebee42d4eefa64c057e53ae441a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:05.329985 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0172ab2e-04c8-4f00-8cf0-43f1bd090f16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1feeebee42d4eefa64c057e53ae441a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:05.332082 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0172ab2e-04c8-4f00-8cf0-43f1bd090f16 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:05.443042 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0172ab2e-04c8-4f00-8cf0-43f1bd090f16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1feeebee42d4eefa64c057e53ae441a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:05.464699 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0172ab2e-04c8-4f00-8cf0-43f1bd090f16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1feeebee42d4eefa64c057e53ae441a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:05.476614 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0172ab2e-04c8-4f00-8cf0-43f1bd090f16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1feeebee42d4eefa64c057e53ae441a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:05.478273 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0172ab2e-04c8-4f00-8cf0-43f1bd090f16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1feeebee42d4eefa64c057e53ae441a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:05.488164 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734077f6-425e-4c0f-94e4-c3bc68e1533d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:05.489470 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734077f6-425e-4c0f-94e4-c3bc68e1533d service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:05.508696 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 984/1972] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:26:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:26:05.510265 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734077f6-425e-4c0f-94e4-c3bc68e1533d service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:05.516530 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0172ab2e-04c8-4f00-8cf0-43f1bd090f16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1feeebee42d4eefa64c057e53ae441a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:05.550900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734077f6-425e-4c0f-94e4-c3bc68e1533d service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:05.559360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b198d-d0dc-4c70-974e-85287423a670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:05.560965 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b198d-d0dc-4c70-974e-85287423a670 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:05.561571 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b198d-d0dc-4c70-974e-85287423a670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:05.562206 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b198d-d0dc-4c70-974e-85287423a670 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:05.563671 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b198d-d0dc-4c70-974e-85287423a670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:05.568829 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b198d-d0dc-4c70-974e-85287423a670 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:05.590796 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 989/1973] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:26:05 2022] DELETE /identity/v3/users/c1feeebee42d4eefa64c057e53ae441a => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:26:05.592198 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0172ab2e-04c8-4f00-8cf0-43f1bd090f16 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:05.595206 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 985/1974] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:26:05.598055 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e698a-5c6d-4cd7-8976-0b0a68e2c591 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:05.598863 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e698a-5c6d-4cd7-8976-0b0a68e2c591 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:05.599903 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e698a-5c6d-4cd7-8976-0b0a68e2c591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:05.622677 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0172ab2e-04c8-4f00-8cf0-43f1bd090f16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:05.688591 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-362e698a-5c6d-4cd7-8976-0b0a68e2c591 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:05.690506 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:05.694519 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-362e698a-5c6d-4cd7-8976-0b0a68e2c591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:05.848604 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:05.887758 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0172ab2e-04c8-4f00-8cf0-43f1bd090f16 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:05.911050 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a089d-1640-40c6-b64b-f7c46f8c429f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:05.912414 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a089d-1640-40c6-b64b-f7c46f8c429f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:05.912895 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a089d-1640-40c6-b64b-f7c46f8c429f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:05.913352 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a089d-1640-40c6-b64b-f7c46f8c429f admin admin] PATH_INFO: `/v3/projects/88dcf6e9e51c44cc9cabb50d37c5e813` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:05.916689 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a089d-1640-40c6-b64b-f7c46f8c429f admin admin] RBAC: Authorizing `identity:delete_project(project_id=88dcf6e9e51c44cc9cabb50d37c5e813)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:05.921591 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a089d-1640-40c6-b64b-f7c46f8c429f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:05.929510 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 986/1975] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:26:05.931415 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e698a-5c6d-4cd7-8976-0b0a68e2c591 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:05.933870 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-362e698a-5c6d-4cd7-8976-0b0a68e2c591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:05.970129 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e698a-5c6d-4cd7-8976-0b0a68e2c591 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:05.977808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a4827-697c-4650-86e6-9ae781feaf48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:05.979398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a4827-697c-4650-86e6-9ae781feaf48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:05.980375 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a4827-697c-4650-86e6-9ae781feaf48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:05.980855 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a4827-697c-4650-86e6-9ae781feaf48 admin admin] PATH_INFO: `/v3/projects/a84bdd938566421f9f98910d89456e09/users/3f0b2bfc8a9141388f13f0cf8be86791/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:06.022479 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504a4827-697c-4650-86e6-9ae781feaf48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a84bdd938566421f9f98910d89456e09, user_id=3f0b2bfc8a9141388f13f0cf8be86791, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:06.028753 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504a4827-697c-4650-86e6-9ae781feaf48 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:06.060481 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 987/1976] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:26:05 2022] PUT /identity/v3/projects/a84bdd938566421f9f98910d89456e09/users/3f0b2bfc8a9141388f13f0cf8be86791/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:26:06.062727 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a4827-697c-4650-86e6-9ae781feaf48 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:06.065717 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-415a089d-1640-40c6-b64b-f7c46f8c429f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88dcf6e9e51c44cc9cabb50d37c5e813', 'request_id': 'req-415a089d-1640-40c6-b64b-f7c46f8c429f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:06.078067 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-504a4827-697c-4650-86e6-9ae781feaf48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:06.084684 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-415a089d-1640-40c6-b64b-f7c46f8c429f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88dcf6e9e51c44cc9cabb50d37c5e813', 'request_id': 'req-415a089d-1640-40c6-b64b-f7c46f8c429f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:06.111554 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-415a089d-1640-40c6-b64b-f7c46f8c429f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88dcf6e9e51c44cc9cabb50d37c5e813', 'request_id': 'req-415a089d-1640-40c6-b64b-f7c46f8c429f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:06.150696 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-415a089d-1640-40c6-b64b-f7c46f8c429f admin admin] The token cache is being invalidate because project 88dcf6e9e51c44cc9cabb50d37c5e813 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:26:06.151227 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-415a089d-1640-40c6-b64b-f7c46f8c429f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:06.154267 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-415a089d-1640-40c6-b64b-f7c46f8c429f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:06.156478 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 990/1977] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:26:05 2022] DELETE /identity/v3/projects/88dcf6e9e51c44cc9cabb50d37c5e813 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:26:06.157612 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a089d-1640-40c6-b64b-f7c46f8c429f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:06.162438 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-415a089d-1640-40c6-b64b-f7c46f8c429f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:06.221721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a4827-697c-4650-86e6-9ae781feaf48 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:06.228358 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e4e5b-27c8-4a25-9151-3c9c9bfe5276 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:06.229489 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e4e5b-27c8-4a25-9151-3c9c9bfe5276 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:06.229976 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e4e5b-27c8-4a25-9151-3c9c9bfe5276 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:06.230647 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e4e5b-27c8-4a25-9151-3c9c9bfe5276 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:06.233006 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b58e4e5b-27c8-4a25-9151-3c9c9bfe5276 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:06.285273 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a089d-1640-40c6-b64b-f7c46f8c429f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:06.361823 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e4e5b-27c8-4a25-9151-3c9c9bfe5276 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:06.366240 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e4e5b-27c8-4a25-9151-3c9c9bfe5276 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:06.407395 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 988/1978] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:26:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:26:06.408086 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d9573-6bd9-4c17-9f04-7116d9205110 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:06.409344 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d9573-6bd9-4c17-9f04-7116d9205110 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:06.409828 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51c6a3-329d-4235-80db-abd877ccfe49 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:06.410030 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d9573-6bd9-4c17-9f04-7116d9205110 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:06.410285 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51c6a3-329d-4235-80db-abd877ccfe49 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:06.410449 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d9573-6bd9-4c17-9f04-7116d9205110 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:06.410742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51c6a3-329d-4235-80db-abd877ccfe49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:06.413997 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1d7d9573-6bd9-4c17-9f04-7116d9205110 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:06.445334 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:06.488575 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1f51c6a3-329d-4235-80db-abd877ccfe49 None None] MFA Rules not processed for user `3f0b2bfc8a9141388f13f0cf8be86791`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:06.490623 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1f51c6a3-329d-4235-80db-abd877ccfe49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:06.638617 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7d9573-6bd9-4c17-9f04-7116d9205110 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:06.642466 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7d9573-6bd9-4c17-9f04-7116d9205110 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:06.645200 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:06.669978 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 989/1979] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:06 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:26:06.672250 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f710e23-02b8-427e-aedd-3ad5030eb7ee None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:06.673455 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f710e23-02b8-427e-aedd-3ad5030eb7ee None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:06.674129 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f710e23-02b8-427e-aedd-3ad5030eb7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:06.726256 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7f710e23-02b8-427e-aedd-3ad5030eb7ee None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:06.730954 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 991/1980] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:26:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:26:06.731424 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7f710e23-02b8-427e-aedd-3ad5030eb7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:06.732676 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d9573-6bd9-4c17-9f04-7116d9205110 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:06.739409 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1d7d9573-6bd9-4c17-9f04-7116d9205110 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:06.772460 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 990/1981] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:26:06.786566 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b0590-1244-49c9-bd94-8a945fbe2455 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:06.787670 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b0590-1244-49c9-bd94-8a945fbe2455 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:06.788216 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b0590-1244-49c9-bd94-8a945fbe2455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:06.925478 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c59b0590-1244-49c9-bd94-8a945fbe2455 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:06.928464 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c59b0590-1244-49c9-bd94-8a945fbe2455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:06.987852 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 991/1982] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:26:06.990050 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b0590-1244-49c9-bd94-8a945fbe2455 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:07.028889 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b0590-1244-49c9-bd94-8a945fbe2455 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:07.048803 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54d88d-2c6c-421a-9eec-d56090a6b820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:07.050184 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54d88d-2c6c-421a-9eec-d56090a6b820 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:07.050820 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54d88d-2c6c-421a-9eec-d56090a6b820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:07.056465 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54d88d-2c6c-421a-9eec-d56090a6b820 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:07.058280 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c54d88d-2c6c-421a-9eec-d56090a6b820 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:26:07.058982 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c54d88d-2c6c-421a-9eec-d56090a6b820 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:07.062293 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c54d88d-2c6c-421a-9eec-d56090a6b820 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:07.123538 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 992/1983] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:26:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:26:07.130767 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54d88d-2c6c-421a-9eec-d56090a6b820 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:07.158412 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d9573-6bd9-4c17-9f04-7116d9205110 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:07.162307 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54d88d-2c6c-421a-9eec-d56090a6b820 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:07.184863 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938fe4d-384d-4c2e-b8cd-efb6bafe0535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:07.184863 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938fe4d-384d-4c2e-b8cd-efb6bafe0535 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:07.185474 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938fe4d-384d-4c2e-b8cd-efb6bafe0535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:07.186052 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938fe4d-384d-4c2e-b8cd-efb6bafe0535 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:07.187399 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8938fe4d-384d-4c2e-b8cd-efb6bafe0535 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:07.189400 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42560f0f-f8f0-4182-8745-e012b048ad71 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98560723b49940d38a6a5e9cf3f5d3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:07.190532 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42560f0f-f8f0-4182-8745-e012b048ad71 service glance] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:07.190982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42560f0f-f8f0-4182-8745-e012b048ad71 service glance] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:07.191460 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42560f0f-f8f0-4182-8745-e012b048ad71 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:07.194432 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8938fe4d-384d-4c2e-b8cd-efb6bafe0535 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:07.254282 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42560f0f-f8f0-4182-8745-e012b048ad71 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:07.265603 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42560f0f-f8f0-4182-8745-e012b048ad71 service glance] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:07.269012 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 993/1984] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:26:07 2022] POST /identity/v3/projects => generated 375 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:26:07.276486 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938fe4d-384d-4c2e-b8cd-efb6bafe0535 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:07.333070 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 992/1985] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:26:06 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:26:07.515317 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938fe4d-384d-4c2e-b8cd-efb6bafe0535 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:07.547663 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e0e11-6321-477b-a76e-06ebe8ca907b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:07.549992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e0e11-6321-477b-a76e-06ebe8ca907b admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:07.556465 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e0e11-6321-477b-a76e-06ebe8ca907b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:07.557549 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e0e11-6321-477b-a76e-06ebe8ca907b admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:07.558797 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e0e11-6321-477b-a76e-06ebe8ca907b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:07.562676 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e0e11-6321-477b-a76e-06ebe8ca907b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:07.647228 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 994/1986] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:26:07 2022] POST /identity/v3/users => generated 288 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:26:07.649785 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42560f0f-f8f0-4182-8745-e012b048ad71 service glance] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:07.676551 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42560f0f-f8f0-4182-8745-e012b048ad71 service glance] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:07.686345 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c218f-47f4-49bf-9568-57f86788fc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:07.687240 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c218f-47f4-49bf-9568-57f86788fc43 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:07.687739 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c218f-47f4-49bf-9568-57f86788fc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:07.688156 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c218f-47f4-49bf-9568-57f86788fc43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:07.689422 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c218f-47f4-49bf-9568-57f86788fc43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:07.692397 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c218f-47f4-49bf-9568-57f86788fc43 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:07.713913 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 993/1987] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:26:07.720318 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e0e11-6321-477b-a76e-06ebe8ca907b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:07.759239 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e0e11-6321-477b-a76e-06ebe8ca907b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:07.783563 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26d80b-2e7c-45e6-b196-d67103ceddb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:07.792386 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26d80b-2e7c-45e6-b196-d67103ceddb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:07.793581 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26d80b-2e7c-45e6-b196-d67103ceddb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:07.794403 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26d80b-2e7c-45e6-b196-d67103ceddb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:07.796564 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d26d80b-2e7c-45e6-b196-d67103ceddb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:07.801461 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d26d80b-2e7c-45e6-b196-d67103ceddb4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:07.865610 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 995/1988] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:26:07.867632 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c218f-47f4-49bf-9568-57f86788fc43 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:07.890486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c218f-47f4-49bf-9568-57f86788fc43 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:07.900811 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c570a-75e5-4b0b-8474-c3da0f41b566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:07.902338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c570a-75e5-4b0b-8474-c3da0f41b566 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:07.902916 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c570a-75e5-4b0b-8474-c3da0f41b566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:07.903389 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c570a-75e5-4b0b-8474-c3da0f41b566 admin admin] PATH_INFO: `/v3/projects/956856bd0f3f4b2a90a2ea664e556920/users/33474f5c8e93483f94ea7ffab006dd50/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:07.954252 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4c570a-75e5-4b0b-8474-c3da0f41b566 admin admin] RBAC: Authorizing `identity:create_grant(project_id=956856bd0f3f4b2a90a2ea664e556920, user_id=33474f5c8e93483f94ea7ffab006dd50, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:07.963153 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4c570a-75e5-4b0b-8474-c3da0f41b566 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:08.004878 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 994/1989] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:26:07 2022] PUT /identity/v3/projects/956856bd0f3f4b2a90a2ea664e556920/users/33474f5c8e93483f94ea7ffab006dd50/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:26:08.017842 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0073b-3806-47e3-bbba-27a4e9f41aed None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:08.018657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0073b-3806-47e3-bbba-27a4e9f41aed None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:08.019138 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0073b-3806-47e3-bbba-27a4e9f41aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:08.088721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-29a0073b-3806-47e3-bbba-27a4e9f41aed None None] MFA Rules not processed for user `33474f5c8e93483f94ea7ffab006dd50`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:08.092230 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-29a0073b-3806-47e3-bbba-27a4e9f41aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:08.292979 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 996/1990] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:26:08.315048 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c570a-75e5-4b0b-8474-c3da0f41b566 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:08.317604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bf4c570a-75e5-4b0b-8474-c3da0f41b566 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:08.515990 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a0073b-3806-47e3-bbba-27a4e9f41aed None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:08.518238 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-29a0073b-3806-47e3-bbba-27a4e9f41aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:08.544489 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c570a-75e5-4b0b-8474-c3da0f41b566 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:08.552151 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ee4d0-3808-4131-af65-ce09263c4169 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:08.553530 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ee4d0-3808-4131-af65-ce09263c4169 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:08.554100 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ee4d0-3808-4131-af65-ce09263c4169 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:08.555362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ee4d0-3808-4131-af65-ce09263c4169 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:08.568966 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:08.575009 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ee4d0-3808-4131-af65-ce09263c4169 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:08.580328 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ee4d0-3808-4131-af65-ce09263c4169 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:08.594713 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:08.628279 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 995/1991] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:26:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:26:08.748497 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a0073b-3806-47e3-bbba-27a4e9f41aed None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:08.754301 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6cb47e-75b2-45a7-bcc9-246c73759008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:08.755344 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6cb47e-75b2-45a7-bcc9-246c73759008 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:08.755753 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6cb47e-75b2-45a7-bcc9-246c73759008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:08.756055 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6cb47e-75b2-45a7-bcc9-246c73759008 admin admin] PATH_INFO: `/v3/projects/2dc7c299d00748aba3ebc7913277477b` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:08.761666 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6cb47e-75b2-45a7-bcc9-246c73759008 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dc7c299d00748aba3ebc7913277477b)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:08.767103 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6cb47e-75b2-45a7-bcc9-246c73759008 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:09.000663 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ec6cb47e-75b2-45a7-bcc9-246c73759008 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dc7c299d00748aba3ebc7913277477b', 'request_id': 'req-ec6cb47e-75b2-45a7-bcc9-246c73759008'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:09.015144 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ec6cb47e-75b2-45a7-bcc9-246c73759008 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dc7c299d00748aba3ebc7913277477b', 'request_id': 'req-ec6cb47e-75b2-45a7-bcc9-246c73759008'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:09.031588 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ec6cb47e-75b2-45a7-bcc9-246c73759008 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dc7c299d00748aba3ebc7913277477b', 'request_id': 'req-ec6cb47e-75b2-45a7-bcc9-246c73759008'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:09.079248 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ec6cb47e-75b2-45a7-bcc9-246c73759008 admin admin] The token cache is being invalidate because project 2dc7c299d00748aba3ebc7913277477b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:26:09.080234 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ec6cb47e-75b2-45a7-bcc9-246c73759008 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:09.081330 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ec6cb47e-75b2-45a7-bcc9-246c73759008 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:09.084497 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 997/1992] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:26:08 2022] DELETE /identity/v3/projects/2dc7c299d00748aba3ebc7913277477b => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:26:09.091919 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ee4d0-3808-4131-af65-ce09263c4169 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:09.126763 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ee4d0-3808-4131-af65-ce09263c4169 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:09.477287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778a6f3-8f40-414b-8b7a-fecbe142d305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:09.480824 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778a6f3-8f40-414b-8b7a-fecbe142d305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:09.481340 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778a6f3-8f40-414b-8b7a-fecbe142d305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:09.481864 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778a6f3-8f40-414b-8b7a-fecbe142d305 admin admin] PATH_INFO: `/v3/users/75bc08c45ea94c6f9f3f663cf705b15b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:09.498175 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d778a6f3-8f40-414b-8b7a-fecbe142d305 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75bc08c45ea94c6f9f3f663cf705b15b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:09.504984 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d778a6f3-8f40-414b-8b7a-fecbe142d305 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:09.692156 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d778a6f3-8f40-414b-8b7a-fecbe142d305 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75bc08c45ea94c6f9f3f663cf705b15b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:09.708164 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d778a6f3-8f40-414b-8b7a-fecbe142d305 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75bc08c45ea94c6f9f3f663cf705b15b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:09.740568 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d778a6f3-8f40-414b-8b7a-fecbe142d305 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75bc08c45ea94c6f9f3f663cf705b15b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:09.741990 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d778a6f3-8f40-414b-8b7a-fecbe142d305 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75bc08c45ea94c6f9f3f663cf705b15b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:09.782248 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d778a6f3-8f40-414b-8b7a-fecbe142d305 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75bc08c45ea94c6f9f3f663cf705b15b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:09.807711 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 996/1993] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:26:09 2022] DELETE /identity/v3/users/75bc08c45ea94c6f9f3f663cf705b15b => generated 0 bytes in 713 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:26:10.425663 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6cb47e-75b2-45a7-bcc9-246c73759008 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:10.428214 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ec6cb47e-75b2-45a7-bcc9-246c73759008 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:10.481091 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6cb47e-75b2-45a7-bcc9-246c73759008 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:10.538894 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778a6f3-8f40-414b-8b7a-fecbe142d305 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:10.541781 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d778a6f3-8f40-414b-8b7a-fecbe142d305 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:10.629898 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3e8c7-9969-4df8-a2b6-318989f90315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:10.632064 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3e8c7-9969-4df8-a2b6-318989f90315 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:10.633797 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3e8c7-9969-4df8-a2b6-318989f90315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:10.634624 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3e8c7-9969-4df8-a2b6-318989f90315 admin admin] PATH_INFO: `/v3/projects/d8039387a2c6432393389b67aff32bfa` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:10.641907 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3e8c7-9969-4df8-a2b6-318989f90315 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8039387a2c6432393389b67aff32bfa)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:10.645187 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3e8c7-9969-4df8-a2b6-318989f90315 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:10.729629 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778a6f3-8f40-414b-8b7a-fecbe142d305 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:10.745086 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d00ed-9baa-4a2e-bb6f-58ad9fdfb9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:10.749148 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d00ed-9baa-4a2e-bb6f-58ad9fdfb9b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:10.750144 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d00ed-9baa-4a2e-bb6f-58ad9fdfb9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:10.751284 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d00ed-9baa-4a2e-bb6f-58ad9fdfb9b9 admin admin] PATH_INFO: `/v3/users/5ca77871109e4e1da54b8b7e0a3fd049` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:10.756227 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d00ed-9baa-4a2e-bb6f-58ad9fdfb9b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ca77871109e4e1da54b8b7e0a3fd049)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:10.763968 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d00ed-9baa-4a2e-bb6f-58ad9fdfb9b9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:10.829096 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:10.830956 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:10.835607 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-abf3e8c7-9969-4df8-a2b6-318989f90315 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8039387a2c6432393389b67aff32bfa', 'request_id': 'req-abf3e8c7-9969-4df8-a2b6-318989f90315'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:10.860790 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-abf3e8c7-9969-4df8-a2b6-318989f90315 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8039387a2c6432393389b67aff32bfa', 'request_id': 'req-abf3e8c7-9969-4df8-a2b6-318989f90315'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:10.890795 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-abf3e8c7-9969-4df8-a2b6-318989f90315 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8039387a2c6432393389b67aff32bfa', 'request_id': 'req-abf3e8c7-9969-4df8-a2b6-318989f90315'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:10.918454 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-507d00ed-9baa-4a2e-bb6f-58ad9fdfb9b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ca77871109e4e1da54b8b7e0a3fd049'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:10.935354 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-abf3e8c7-9969-4df8-a2b6-318989f90315 admin admin] The token cache is being invalidate because project d8039387a2c6432393389b67aff32bfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:26:10.935869 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-507d00ed-9baa-4a2e-bb6f-58ad9fdfb9b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ca77871109e4e1da54b8b7e0a3fd049'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:10.935983 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-abf3e8c7-9969-4df8-a2b6-318989f90315 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:10.937261 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-abf3e8c7-9969-4df8-a2b6-318989f90315 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:10.947627 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 998/1994] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:26:10 2022] DELETE /identity/v3/projects/d8039387a2c6432393389b67aff32bfa => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:26:10.950631 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649664e1-df87-4878-97a6-95982a9bad0e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:10.952278 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649664e1-df87-4878-97a6-95982a9bad0e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:10.952761 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649664e1-df87-4878-97a6-95982a9bad0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:10.973974 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-507d00ed-9baa-4a2e-bb6f-58ad9fdfb9b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ca77871109e4e1da54b8b7e0a3fd049'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:10.975177 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-507d00ed-9baa-4a2e-bb6f-58ad9fdfb9b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ca77871109e4e1da54b8b7e0a3fd049'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:11.037230 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-507d00ed-9baa-4a2e-bb6f-58ad9fdfb9b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ca77871109e4e1da54b8b7e0a3fd049'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:11.047685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-649664e1-df87-4878-97a6-95982a9bad0e None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:11.050600 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-649664e1-df87-4878-97a6-95982a9bad0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:11.052558 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 997/1995] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:26:10 2022] DELETE /identity/v3/users/5ca77871109e4e1da54b8b7e0a3fd049 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:26:11.059194 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c8d2a-9d55-485c-8b3d-826d149c6ad7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:11.059487 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c8d2a-9d55-485c-8b3d-826d149c6ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:11.059868 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c8d2a-9d55-485c-8b3d-826d149c6ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:11.115651 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6e5c8d2a-9d55-485c-8b3d-826d149c6ad7 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:11.116997 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6e5c8d2a-9d55-485c-8b3d-826d149c6ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:11.377750 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 998/1996] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:26:11.392385 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 999/1997] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 444 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:26:11.403878 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c8d2a-9d55-485c-8b3d-826d149c6ad7 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:11.408070 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718dac0-941e-459e-87d5-b4fb7091b7ab None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:11.408703 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718dac0-941e-459e-87d5-b4fb7091b7ab None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:11.409615 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718dac0-941e-459e-87d5-b4fb7091b7ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:11.412170 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6e5c8d2a-9d55-485c-8b3d-826d149c6ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:11.492886 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9718dac0-941e-459e-87d5-b4fb7091b7ab None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:11.492886 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9718dac0-941e-459e-87d5-b4fb7091b7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:11.509905 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1000/1998] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:26:11.516452 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718dac0-941e-459e-87d5-b4fb7091b7ab None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:11.529926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718dac0-941e-459e-87d5-b4fb7091b7ab None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:11.536294 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f735d0ae-c20e-41f5-bcfe-0d03f536daa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:11.537412 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735d0ae-c20e-41f5-bcfe-0d03f536daa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:11.537886 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735d0ae-c20e-41f5-bcfe-0d03f536daa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:11.539561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735d0ae-c20e-41f5-bcfe-0d03f536daa4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:11.542407 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f735d0ae-c20e-41f5-bcfe-0d03f536daa4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:26:11.542961 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f735d0ae-c20e-41f5-bcfe-0d03f536daa4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:11.545765 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f735d0ae-c20e-41f5-bcfe-0d03f536daa4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:11.583430 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c8d2a-9d55-485c-8b3d-826d149c6ad7 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:11.593284 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98098f5-9229-4a93-8ced-ff4d25d5b9ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:11.594779 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1001/1999] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:26:11.595352 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98098f5-9229-4a93-8ced-ff4d25d5b9ef service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:11.596333 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98098f5-9229-4a93-8ced-ff4d25d5b9ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:11.597107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98098f5-9229-4a93-8ced-ff4d25d5b9ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:11.603215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f735d0ae-c20e-41f5-bcfe-0d03f536daa4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:11.632268 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f735d0ae-c20e-41f5-bcfe-0d03f536daa4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:11.642400 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96fa12-5ff8-430f-9e23-7dcfb6f2b472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:11.643489 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96fa12-5ff8-430f-9e23-7dcfb6f2b472 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:11.644238 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96fa12-5ff8-430f-9e23-7dcfb6f2b472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:11.645196 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96fa12-5ff8-430f-9e23-7dcfb6f2b472 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:11.647022 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a96fa12-5ff8-430f-9e23-7dcfb6f2b472 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:11.652212 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a96fa12-5ff8-430f-9e23-7dcfb6f2b472 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:11.674011 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98098f5-9229-4a93-8ced-ff4d25d5b9ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:11.689093 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98098f5-9229-4a93-8ced-ff4d25d5b9ef service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:11.736178 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 999/2000] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:26:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:26:11.772168 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1002/2001] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:26:11 2022] POST /identity/v3/projects => generated 381 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:26:11.777746 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98098f5-9229-4a93-8ced-ff4d25d5b9ef service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:11.852565 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98098f5-9229-4a93-8ced-ff4d25d5b9ef service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:11.984448 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:12.058359 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd371d2-9d57-4b33-9730-e2dbe1121c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:12.060441 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd371d2-9d57-4b33-9730-e2dbe1121c7e admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:12.060441 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd371d2-9d57-4b33-9730-e2dbe1121c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:12.060441 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd371d2-9d57-4b33-9730-e2dbe1121c7e admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:12.061272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd371d2-9d57-4b33-9730-e2dbe1121c7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:12.072554 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd371d2-9d57-4b33-9730-e2dbe1121c7e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:12.165336 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1000/2002] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:26:11 2022] POST /identity/v3/users => generated 291 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:26:12.176147 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96fa12-5ff8-430f-9e23-7dcfb6f2b472 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:12.198846 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96fa12-5ff8-430f-9e23-7dcfb6f2b472 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:12.218914 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b3555-2f72-412e-ad74-107d3ada35de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:12.219699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b3555-2f72-412e-ad74-107d3ada35de admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:12.220093 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b3555-2f72-412e-ad74-107d3ada35de admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:12.220356 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b3555-2f72-412e-ad74-107d3ada35de admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:12.221165 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5b3555-2f72-412e-ad74-107d3ada35de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:12.222963 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5b3555-2f72-412e-ad74-107d3ada35de admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:12.247170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1003/2003] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:26:12.248649 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd371d2-9d57-4b33-9730-e2dbe1121c7e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:12.283048 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd371d2-9d57-4b33-9730-e2dbe1121c7e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:12.314130 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09893359-dd3a-4f27-88f3-cd56774c41a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:12.318347 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09893359-dd3a-4f27-88f3-cd56774c41a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:12.319600 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09893359-dd3a-4f27-88f3-cd56774c41a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:12.320091 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09893359-dd3a-4f27-88f3-cd56774c41a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:12.322490 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09893359-dd3a-4f27-88f3-cd56774c41a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:12.341350 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09893359-dd3a-4f27-88f3-cd56774c41a7 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:12.407180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1001/2004] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:26:12.420363 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b3555-2f72-412e-ad74-107d3ada35de admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:12.534597 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b3555-2f72-412e-ad74-107d3ada35de admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:12.559477 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf15ff-72f1-46d0-8174-b5c2a5ff3d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:12.590133 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf15ff-72f1-46d0-8174-b5c2a5ff3d9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:12.595317 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf15ff-72f1-46d0-8174-b5c2a5ff3d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:12.596273 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf15ff-72f1-46d0-8174-b5c2a5ff3d9c admin admin] PATH_INFO: `/v3/projects/208f76a478cf4467a0a41f26ef57bf7b/users/1c19857e7eee4fffa63f6ba507e119cf/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:12.673935 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf15ff-72f1-46d0-8174-b5c2a5ff3d9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=208f76a478cf4467a0a41f26ef57bf7b, user_id=1c19857e7eee4fffa63f6ba507e119cf, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:12.680360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf15ff-72f1-46d0-8174-b5c2a5ff3d9c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:12.724062 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1004/2005] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:26:12 2022] PUT /identity/v3/projects/208f76a478cf4467a0a41f26ef57bf7b/users/1c19857e7eee4fffa63f6ba507e119cf/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:26:12.745303 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdcf168-6803-4645-b895-7ebcc0a5b5b2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:12.746214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdcf168-6803-4645-b895-7ebcc0a5b5b2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:12.746775 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdcf168-6803-4645-b895-7ebcc0a5b5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:12.771963 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf15ff-72f1-46d0-8174-b5c2a5ff3d9c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:12.777235 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9ddf15ff-72f1-46d0-8174-b5c2a5ff3d9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:12.924300 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5fdcf168-6803-4645-b895-7ebcc0a5b5b2 None None] MFA Rules not processed for user `1c19857e7eee4fffa63f6ba507e119cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:12.926077 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5fdcf168-6803-4645-b895-7ebcc0a5b5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:13.152102 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf15ff-72f1-46d0-8174-b5c2a5ff3d9c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:13.169585 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1002/2006] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:26:13.171840 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdcf168-6803-4645-b895-7ebcc0a5b5b2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:13.183857 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f91ebb-91d5-495a-9359-254e45980aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:13.183857 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f91ebb-91d5-495a-9359-254e45980aa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:13.183857 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f91ebb-91d5-495a-9359-254e45980aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:13.183857 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f91ebb-91d5-495a-9359-254e45980aa5 admin admin] PATH_INFO: `/v3/projects/36eea90f360646829dad58a5e15e9338` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:13.185461 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f91ebb-91d5-495a-9359-254e45980aa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36eea90f360646829dad58a5e15e9338)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:13.196428 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f91ebb-91d5-495a-9359-254e45980aa5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:13.204070 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:13.213743 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdcf168-6803-4645-b895-7ebcc0a5b5b2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:13.257587 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f50455-777d-40e8-a0ca-b4a6d2c13b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:13.267209 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f50455-777d-40e8-a0ca-b4a6d2c13b35 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:13.268785 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f50455-777d-40e8-a0ca-b4a6d2c13b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:13.269354 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f50455-777d-40e8-a0ca-b4a6d2c13b35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:13.270670 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f50455-777d-40e8-a0ca-b4a6d2c13b35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:13.283776 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f50455-777d-40e8-a0ca-b4a6d2c13b35 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:13.315906 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-26f91ebb-91d5-495a-9359-254e45980aa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36eea90f360646829dad58a5e15e9338', 'request_id': 'req-26f91ebb-91d5-495a-9359-254e45980aa5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:13.331937 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-26f91ebb-91d5-495a-9359-254e45980aa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36eea90f360646829dad58a5e15e9338', 'request_id': 'req-26f91ebb-91d5-495a-9359-254e45980aa5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:13.346078 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-26f91ebb-91d5-495a-9359-254e45980aa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36eea90f360646829dad58a5e15e9338', 'request_id': 'req-26f91ebb-91d5-495a-9359-254e45980aa5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:13.376314 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-26f91ebb-91d5-495a-9359-254e45980aa5 admin admin] The token cache is being invalidate because project 36eea90f360646829dad58a5e15e9338 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:26:13.377037 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-26f91ebb-91d5-495a-9359-254e45980aa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:13.378612 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-26f91ebb-91d5-495a-9359-254e45980aa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:13.382702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1005/2007] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:26:12 2022] DELETE /identity/v3/projects/36eea90f360646829dad58a5e15e9338 => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:26:13.390272 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f91ebb-91d5-495a-9359-254e45980aa5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:13.394804 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-26f91ebb-91d5-495a-9359-254e45980aa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:13.460682 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1003/2008] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:26:13 2022] POST /identity/v3/projects => generated 381 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:26:13.465459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f50455-777d-40e8-a0ca-b4a6d2c13b35 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:13.479662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a3f50455-777d-40e8-a0ca-b4a6d2c13b35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:13.716462 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f91ebb-91d5-495a-9359-254e45980aa5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:13.740532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0510f5a2-db16-454a-bb2e-2a891bb2aad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:13.743152 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510f5a2-db16-454a-bb2e-2a891bb2aad1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:13.755797 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510f5a2-db16-454a-bb2e-2a891bb2aad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:13.755797 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510f5a2-db16-454a-bb2e-2a891bb2aad1 admin admin] PATH_INFO: `/v3/users/db8808aacd7b4f789bc59944e9ff0499` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:13.755797 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0510f5a2-db16-454a-bb2e-2a891bb2aad1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db8808aacd7b4f789bc59944e9ff0499)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:13.756511 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f50455-777d-40e8-a0ca-b4a6d2c13b35 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:13.757195 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0510f5a2-db16-454a-bb2e-2a891bb2aad1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:13.787280 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d06b4-25f8-4833-be50-3b922975059b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:13.792046 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d06b4-25f8-4833-be50-3b922975059b admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:13.792882 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d06b4-25f8-4833-be50-3b922975059b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:13.796019 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d06b4-25f8-4833-be50-3b922975059b admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:13.797724 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d06b4-25f8-4833-be50-3b922975059b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:13.814300 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d06b4-25f8-4833-be50-3b922975059b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:13.877931 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1004/2009] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:26:13 2022] POST /identity/v3/users => generated 297 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:26:13.885272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d06b4-25f8-4833-be50-3b922975059b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:13.931246 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d06b4-25f8-4833-be50-3b922975059b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:13.944536 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f102e1-345c-47b2-a143-1d978c232bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:13.946970 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f102e1-345c-47b2-a143-1d978c232bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:13.948315 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f102e1-345c-47b2-a143-1d978c232bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:13.949945 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f102e1-345c-47b2-a143-1d978c232bc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:13.952579 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f102e1-345c-47b2-a143-1d978c232bc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:13.955789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f102e1-345c-47b2-a143-1d978c232bc6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:13.958560 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0510f5a2-db16-454a-bb2e-2a891bb2aad1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db8808aacd7b4f789bc59944e9ff0499'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:13.973431 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1005/2010] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:26:13.980803 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f102e1-345c-47b2-a143-1d978c232bc6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:13.986284 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0510f5a2-db16-454a-bb2e-2a891bb2aad1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db8808aacd7b4f789bc59944e9ff0499'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:14.004249 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0510f5a2-db16-454a-bb2e-2a891bb2aad1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db8808aacd7b4f789bc59944e9ff0499'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:14.005903 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0510f5a2-db16-454a-bb2e-2a891bb2aad1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db8808aacd7b4f789bc59944e9ff0499'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:14.006955 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f102e1-345c-47b2-a143-1d978c232bc6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:14.007283 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0510f5a2-db16-454a-bb2e-2a891bb2aad1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db8808aacd7b4f789bc59944e9ff0499'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:14.017979 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07689f70-ea73-4e0f-91c4-109c0dabb30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:14.019866 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07689f70-ea73-4e0f-91c4-109c0dabb30d admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:14.020105 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07689f70-ea73-4e0f-91c4-109c0dabb30d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:14.021458 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07689f70-ea73-4e0f-91c4-109c0dabb30d admin admin] PATH_INFO: `/v3/projects/ab9fc4f5e92a4f53a1bcbc00928e3784/users/ee1241fbe2f448a6a7fc5969e4b0266e/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:14.027379 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:14.054990 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1006/2011] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:26:13 2022] DELETE /identity/v3/users/db8808aacd7b4f789bc59944e9ff0499 => generated 0 bytes in 662 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:26:14.059733 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07689f70-ea73-4e0f-91c4-109c0dabb30d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab9fc4f5e92a4f53a1bcbc00928e3784, user_id=ee1241fbe2f448a6a7fc5969e4b0266e, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:14.065443 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07689f70-ea73-4e0f-91c4-109c0dabb30d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:14.099724 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1006/2012] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:26:13 2022] PUT /identity/v3/projects/ab9fc4f5e92a4f53a1bcbc00928e3784/users/ee1241fbe2f448a6a7fc5969e4b0266e/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:26:14.113242 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0510f5a2-db16-454a-bb2e-2a891bb2aad1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:14.117053 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0510f5a2-db16-454a-bb2e-2a891bb2aad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:14.145301 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:14.278456 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07689f70-ea73-4e0f-91c4-109c0dabb30d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:14.282823 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-07689f70-ea73-4e0f-91c4-109c0dabb30d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:14.314776 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0510f5a2-db16-454a-bb2e-2a891bb2aad1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:14.518090 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ad091-1fcd-48c5-b03a-41bf9f930cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:14.518865 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ad091-1fcd-48c5-b03a-41bf9f930cbe admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:14.519257 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ad091-1fcd-48c5-b03a-41bf9f930cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:14.519562 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ad091-1fcd-48c5-b03a-41bf9f930cbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:14.520994 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ad091-1fcd-48c5-b03a-41bf9f930cbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:14.522162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07689f70-ea73-4e0f-91c4-109c0dabb30d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:14.525436 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ad091-1fcd-48c5-b03a-41bf9f930cbe admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:14.536695 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f4e0c-54e0-4578-b64b-26e0f777ecb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:14.540734 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f4e0c-54e0-4578-b64b-26e0f777ecb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:14.543575 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f4e0c-54e0-4578-b64b-26e0f777ecb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:14.549330 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1007/2013] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 434 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:26:14.550397 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f4e0c-54e0-4578-b64b-26e0f777ecb9 admin admin] PATH_INFO: `/v3/users/33474f5c8e93483f94ea7ffab006dd50` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:14.550901 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ad091-1fcd-48c5-b03a-41bf9f930cbe admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:14.553362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871f4e0c-54e0-4578-b64b-26e0f777ecb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33474f5c8e93483f94ea7ffab006dd50)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:14.563578 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871f4e0c-54e0-4578-b64b-26e0f777ecb9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:14.570264 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ad091-1fcd-48c5-b03a-41bf9f930cbe admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:14.578485 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2681e0a-9d47-45b5-9281-1bef456bc4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:14.580472 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2681e0a-9d47-45b5-9281-1bef456bc4fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:14.580897 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2681e0a-9d47-45b5-9281-1bef456bc4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:14.581213 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2681e0a-9d47-45b5-9281-1bef456bc4fe admin admin] PATH_INFO: `/v3/projects/ab9fc4f5e92a4f53a1bcbc00928e3784/users/ee1241fbe2f448a6a7fc5969e4b0266e/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:14.584209 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2681e0a-9d47-45b5-9281-1bef456bc4fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab9fc4f5e92a4f53a1bcbc00928e3784, user_id=ee1241fbe2f448a6a7fc5969e4b0266e, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:14.590968 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2681e0a-9d47-45b5-9281-1bef456bc4fe admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:14.623551 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1008/2014] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:26:14 2022] PUT /identity/v3/projects/ab9fc4f5e92a4f53a1bcbc00928e3784/users/ee1241fbe2f448a6a7fc5969e4b0266e/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:26:14.632407 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea1d96-6539-4938-b98c-c8e91b556033 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:14.632781 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea1d96-6539-4938-b98c-c8e91b556033 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:14.633242 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea1d96-6539-4938-b98c-c8e91b556033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:14.688706 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-40ea1d96-6539-4938-b98c-c8e91b556033 None None] MFA Rules not processed for user `ee1241fbe2f448a6a7fc5969e4b0266e`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:14.690509 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-40ea1d96-6539-4938-b98c-c8e91b556033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:14.714756 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-871f4e0c-54e0-4578-b64b-26e0f777ecb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33474f5c8e93483f94ea7ffab006dd50'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:14.737822 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-871f4e0c-54e0-4578-b64b-26e0f777ecb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33474f5c8e93483f94ea7ffab006dd50'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:14.780702 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-871f4e0c-54e0-4578-b64b-26e0f777ecb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33474f5c8e93483f94ea7ffab006dd50'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:14.782302 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-871f4e0c-54e0-4578-b64b-26e0f777ecb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33474f5c8e93483f94ea7ffab006dd50'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:14.819078 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-871f4e0c-54e0-4578-b64b-26e0f777ecb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33474f5c8e93483f94ea7ffab006dd50'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:14.834039 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1007/2015] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:26:14 2022] DELETE /identity/v3/users/33474f5c8e93483f94ea7ffab006dd50 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:26:14.840695 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f4e0c-54e0-4578-b64b-26e0f777ecb9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:14.843359 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-871f4e0c-54e0-4578-b64b-26e0f777ecb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:14.844492 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1009/2016] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:14 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:26:14.846245 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee00ad2-c49e-452c-a56e-8a879857a7e4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:14.846643 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee00ad2-c49e-452c-a56e-8a879857a7e4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:14.847159 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee00ad2-c49e-452c-a56e-8a879857a7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:14.906146 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-eee00ad2-c49e-452c-a56e-8a879857a7e4 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:14.908415 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-eee00ad2-c49e-452c-a56e-8a879857a7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:14.954779 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:15.016445 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f4e0c-54e0-4578-b64b-26e0f777ecb9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:15.037824 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5ca2b-deac-4883-bcbd-25a9034bc62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:15.039810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5ca2b-deac-4883-bcbd-25a9034bc62c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:15.040241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5ca2b-deac-4883-bcbd-25a9034bc62c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:15.041327 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5ca2b-deac-4883-bcbd-25a9034bc62c admin admin] PATH_INFO: `/v3/projects/f93ae75cfc5f48b383cea98cf7f8fcc0` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:15.045742 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc5ca2b-deac-4883-bcbd-25a9034bc62c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f93ae75cfc5f48b383cea98cf7f8fcc0)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:15.048034 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc5ca2b-deac-4883-bcbd-25a9034bc62c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:15.051049 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:15.136641 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1010/2017] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:26:15.138862 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee00ad2-c49e-452c-a56e-8a879857a7e4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:15.172483 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-eee00ad2-c49e-452c-a56e-8a879857a7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:15.218355 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0bc5ca2b-deac-4883-bcbd-25a9034bc62c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f93ae75cfc5f48b383cea98cf7f8fcc0', 'request_id': 'req-0bc5ca2b-deac-4883-bcbd-25a9034bc62c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:15.237370 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0bc5ca2b-deac-4883-bcbd-25a9034bc62c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f93ae75cfc5f48b383cea98cf7f8fcc0', 'request_id': 'req-0bc5ca2b-deac-4883-bcbd-25a9034bc62c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:15.263602 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0bc5ca2b-deac-4883-bcbd-25a9034bc62c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f93ae75cfc5f48b383cea98cf7f8fcc0', 'request_id': 'req-0bc5ca2b-deac-4883-bcbd-25a9034bc62c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:15.306245 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0bc5ca2b-deac-4883-bcbd-25a9034bc62c admin admin] The token cache is being invalidate because project f93ae75cfc5f48b383cea98cf7f8fcc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:26:15.317972 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0bc5ca2b-deac-4883-bcbd-25a9034bc62c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:15.321820 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0bc5ca2b-deac-4883-bcbd-25a9034bc62c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:15.331662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1008/2018] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:26:14 2022] DELETE /identity/v3/projects/f93ae75cfc5f48b383cea98cf7f8fcc0 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:26:15.334029 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5ca2b-deac-4883-bcbd-25a9034bc62c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:15.337198 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0bc5ca2b-deac-4883-bcbd-25a9034bc62c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:15.368423 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee00ad2-c49e-452c-a56e-8a879857a7e4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:15.488658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c76b2a-de4f-4ff9-9186-59290f524e81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:15.491171 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c76b2a-de4f-4ff9-9186-59290f524e81 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:15.491474 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c76b2a-de4f-4ff9-9186-59290f524e81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:15.491474 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c76b2a-de4f-4ff9-9186-59290f524e81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:15.502453 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-36c76b2a-de4f-4ff9-9186-59290f524e81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:15.526235 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5ca2b-deac-4883-bcbd-25a9034bc62c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:15.534062 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93bb79-db73-4682-87bf-03bba2c4323c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:15.535559 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93bb79-db73-4682-87bf-03bba2c4323c service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:15.536257 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93bb79-db73-4682-87bf-03bba2c4323c service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:15.536818 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93bb79-db73-4682-87bf-03bba2c4323c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:15.541676 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ee93bb79-db73-4682-87bf-03bba2c4323c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:15.653253 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c76b2a-de4f-4ff9-9186-59290f524e81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:15.654971 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c76b2a-de4f-4ff9-9186-59290f524e81 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:15.661110 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93bb79-db73-4682-87bf-03bba2c4323c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:15.667419 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93bb79-db73-4682-87bf-03bba2c4323c service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:15.668454 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1011/2019] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:26:15 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:26:15.691053 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d55e4-b755-46c2-8fd3-7aa230e79ae9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:15.691053 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d55e4-b755-46c2-8fd3-7aa230e79ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:15.691053 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d55e4-b755-46c2-8fd3-7aa230e79ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:15.697996 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1009/2020] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:26:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:26:15.787658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b50d55e4-b755-46c2-8fd3-7aa230e79ae9 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:15.789604 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b50d55e4-b755-46c2-8fd3-7aa230e79ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:15.833207 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1012/2021] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:26:15.846734 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b95ade-7cc1-4b0c-9175-d9d8d47df0eb None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:15.848957 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b95ade-7cc1-4b0c-9175-d9d8d47df0eb None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:15.849513 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b95ade-7cc1-4b0c-9175-d9d8d47df0eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:15.860658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50d55e4-b755-46c2-8fd3-7aa230e79ae9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:15.862218 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b50d55e4-b755-46c2-8fd3-7aa230e79ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:15.886608 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50d55e4-b755-46c2-8fd3-7aa230e79ae9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:15.901190 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d876f2-3918-4adb-a797-47b884671740 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:15.904284 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d876f2-3918-4adb-a797-47b884671740 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:15.904713 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d876f2-3918-4adb-a797-47b884671740 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:15.905006 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d876f2-3918-4adb-a797-47b884671740 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:15.909668 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-08d876f2-3918-4adb-a797-47b884671740 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:15.931958 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-36b95ade-7cc1-4b0c-9175-d9d8d47df0eb None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:15.940546 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-36b95ade-7cc1-4b0c-9175-d9d8d47df0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:15.950117 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:15.985574 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1010/2022] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:26:15.989453 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b95ade-7cc1-4b0c-9175-d9d8d47df0eb None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:16.013353 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b95ade-7cc1-4b0c-9175-d9d8d47df0eb None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:16.031636 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc14892-fe98-46f9-94f3-c7c071b39755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:16.032870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc14892-fe98-46f9-94f3-c7c071b39755 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:16.033158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc14892-fe98-46f9-94f3-c7c071b39755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:16.033616 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc14892-fe98-46f9-94f3-c7c071b39755 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:16.034447 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc14892-fe98-46f9-94f3-c7c071b39755 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:26:16.034777 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc14892-fe98-46f9-94f3-c7c071b39755 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:16.036789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc14892-fe98-46f9-94f3-c7c071b39755 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:16.037215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d876f2-3918-4adb-a797-47b884671740 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:16.041027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d876f2-3918-4adb-a797-47b884671740 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:16.079296 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1013/2023] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:26:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:26:16.086667 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1011/2024] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:26:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:26:16.090507 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d876f2-3918-4adb-a797-47b884671740 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:16.169259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d876f2-3918-4adb-a797-47b884671740 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:16.183923 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319e8ee5-e94d-424e-a30d-be054378e61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:16.185974 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e8ee5-e94d-424e-a30d-be054378e61c admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:16.186601 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e8ee5-e94d-424e-a30d-be054378e61c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:16.187025 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e8ee5-e94d-424e-a30d-be054378e61c admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:16.188388 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319e8ee5-e94d-424e-a30d-be054378e61c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:16.191626 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319e8ee5-e94d-424e-a30d-be054378e61c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:16.349067 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1014/2025] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:26:16 2022] POST /identity/v3/projects => generated 375 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:26:16.356310 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc14892-fe98-46f9-94f3-c7c071b39755 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:16.380971 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc14892-fe98-46f9-94f3-c7c071b39755 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:16.442030 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319e8ee5-e94d-424e-a30d-be054378e61c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:16.446569 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-319e8ee5-e94d-424e-a30d-be054378e61c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:16.462686 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:16.611009 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065329f-f4ca-43e9-9b23-7f2bf4c7e10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:16.612530 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065329f-f4ca-43e9-9b23-7f2bf4c7e10a admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:16.613874 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065329f-f4ca-43e9-9b23-7f2bf4c7e10a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:16.614671 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065329f-f4ca-43e9-9b23-7f2bf4c7e10a admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:16.616561 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065329f-f4ca-43e9-9b23-7f2bf4c7e10a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:16.623004 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065329f-f4ca-43e9-9b23-7f2bf4c7e10a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:16.658933 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319e8ee5-e94d-424e-a30d-be054378e61c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:16.668229 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74942286-8153-4be5-8540-cb8373523119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:16.670412 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74942286-8153-4be5-8540-cb8373523119 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:16.670738 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74942286-8153-4be5-8540-cb8373523119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:16.671029 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74942286-8153-4be5-8540-cb8373523119 admin admin] PATH_INFO: `/v3/projects/956856bd0f3f4b2a90a2ea664e556920` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:16.673612 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74942286-8153-4be5-8540-cb8373523119 admin admin] RBAC: Authorizing `identity:delete_project(project_id=956856bd0f3f4b2a90a2ea664e556920)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:16.675833 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74942286-8153-4be5-8540-cb8373523119 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:16.694140 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1012/2026] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:26:16 2022] POST /identity/v3/users => generated 288 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:26:16.698374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065329f-f4ca-43e9-9b23-7f2bf4c7e10a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:16.713389 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065329f-f4ca-43e9-9b23-7f2bf4c7e10a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:16.720033 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5e102-410b-4524-84ac-4261898898c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:16.721263 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5e102-410b-4524-84ac-4261898898c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:16.721561 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5e102-410b-4524-84ac-4261898898c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:16.721994 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5e102-410b-4524-84ac-4261898898c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:16.722829 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e5e102-410b-4524-84ac-4261898898c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:16.725118 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e5e102-410b-4524-84ac-4261898898c9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:16.753267 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1013/2027] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:26:16 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:26:16.756783 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5e102-410b-4524-84ac-4261898898c9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:16.784636 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5e102-410b-4524-84ac-4261898898c9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:16.786634 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-74942286-8153-4be5-8540-cb8373523119 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '956856bd0f3f4b2a90a2ea664e556920', 'request_id': 'req-74942286-8153-4be5-8540-cb8373523119'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:16.803795 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-74942286-8153-4be5-8540-cb8373523119 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '956856bd0f3f4b2a90a2ea664e556920', 'request_id': 'req-74942286-8153-4be5-8540-cb8373523119'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:16.821881 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-74942286-8153-4be5-8540-cb8373523119 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '956856bd0f3f4b2a90a2ea664e556920', 'request_id': 'req-74942286-8153-4be5-8540-cb8373523119'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:16.858511 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-74942286-8153-4be5-8540-cb8373523119 admin admin] The token cache is being invalidate because project 956856bd0f3f4b2a90a2ea664e556920 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:26:16.858969 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-74942286-8153-4be5-8540-cb8373523119 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:16.860624 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-74942286-8153-4be5-8540-cb8373523119 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:16.865763 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1015/2028] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:26:16 2022] DELETE /identity/v3/projects/956856bd0f3f4b2a90a2ea664e556920 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:26:16.876750 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59db21-9bd7-4abe-9d6b-91452900848b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:16.877850 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59db21-9bd7-4abe-9d6b-91452900848b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:16.883874 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59db21-9bd7-4abe-9d6b-91452900848b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:16.941249 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0b59db21-9bd7-4abe-9d6b-91452900848b None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:16.942952 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0b59db21-9bd7-4abe-9d6b-91452900848b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:16.953460 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca55bb3-6b4d-428c-adb0-214a0c28f462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:16.955604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca55bb3-6b4d-428c-adb0-214a0c28f462 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:16.955604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca55bb3-6b4d-428c-adb0-214a0c28f462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:16.955862 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca55bb3-6b4d-428c-adb0-214a0c28f462 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:16.957370 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca55bb3-6b4d-428c-adb0-214a0c28f462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:16.960031 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca55bb3-6b4d-428c-adb0-214a0c28f462 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:16.980303 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1014/2029] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:26:16 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:26:16.982265 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca55bb3-6b4d-428c-adb0-214a0c28f462 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:16.984345 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:16.984834 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6ca55bb3-6b4d-428c-adb0-214a0c28f462 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:17.013322 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca55bb3-6b4d-428c-adb0-214a0c28f462 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:17.021715 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beda3ef-ba93-446d-8ee0-13183155f362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:17.023250 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beda3ef-ba93-446d-8ee0-13183155f362 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:17.023806 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beda3ef-ba93-446d-8ee0-13183155f362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:17.024276 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beda3ef-ba93-446d-8ee0-13183155f362 admin admin] PATH_INFO: `/v3/projects/1ee9720cbdf04d17b7c0b08fe373d203/users/defcbd87d502466684e1669e25962aed/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:17.054428 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beda3ef-ba93-446d-8ee0-13183155f362 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ee9720cbdf04d17b7c0b08fe373d203, user_id=defcbd87d502466684e1669e25962aed, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:17.057103 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1016/2030] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:26:17.060132 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beda3ef-ba93-446d-8ee0-13183155f362 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:17.094963 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1015/2031] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:26:16 2022] PUT /identity/v3/projects/1ee9720cbdf04d17b7c0b08fe373d203/users/defcbd87d502466684e1669e25962aed/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:26:17.114415 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd012ea-a981-43dd-860b-27b23332ea68 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:17.118115 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd012ea-a981-43dd-860b-27b23332ea68 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:17.119599 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd012ea-a981-43dd-860b-27b23332ea68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:17.120928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d6df9-41d2-4a4c-a09a-526874df4a6e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:17.121598 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d6df9-41d2-4a4c-a09a-526874df4a6e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:17.122199 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d6df9-41d2-4a4c-a09a-526874df4a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:17.177823 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d96d6df9-41d2-4a4c-a09a-526874df4a6e None None] MFA Rules not processed for user `defcbd87d502466684e1669e25962aed`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:17.179307 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d96d6df9-41d2-4a4c-a09a-526874df4a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:17.198181 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ebd012ea-a981-43dd-860b-27b23332ea68 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:17.203778 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ebd012ea-a981-43dd-860b-27b23332ea68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:17.387567 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1016/2032] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:26:17.407330 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d6df9-41d2-4a4c-a09a-526874df4a6e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:17.413403 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d96d6df9-41d2-4a4c-a09a-526874df4a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:17.426452 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:17.485084 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1017/2033] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:26:17.486880 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd012ea-a981-43dd-860b-27b23332ea68 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:17.511348 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd012ea-a981-43dd-860b-27b23332ea68 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:17.527454 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d6df9-41d2-4a4c-a09a-526874df4a6e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:17.531112 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae63b9a-0c52-41ee-9b38-be7848c657dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:17.532829 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae63b9a-0c52-41ee-9b38-be7848c657dc admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:17.532829 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae63b9a-0c52-41ee-9b38-be7848c657dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:17.532829 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae63b9a-0c52-41ee-9b38-be7848c657dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:17.533923 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae63b9a-0c52-41ee-9b38-be7848c657dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:26:17.535450 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae63b9a-0c52-41ee-9b38-be7848c657dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:17.543409 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae63b9a-0c52-41ee-9b38-be7848c657dc admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:17.546408 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b12ee3-ac72-4fde-a446-b058a24de1b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:17.547677 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b12ee3-ac72-4fde-a446-b058a24de1b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:17.547970 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b12ee3-ac72-4fde-a446-b058a24de1b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:17.548252 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b12ee3-ac72-4fde-a446-b058a24de1b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:17.569787 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b12ee3-ac72-4fde-a446-b058a24de1b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:17.572854 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b12ee3-ac72-4fde-a446-b058a24de1b3 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:17.584840 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1017/2034] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:26:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:26:17.604752 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1018/2035] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:26:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:26:17.610748 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b12ee3-ac72-4fde-a446-b058a24de1b3 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:17.651269 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b12ee3-ac72-4fde-a446-b058a24de1b3 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:17.662586 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c55f8d-6337-4bc9-9c92-17bbc02533e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:17.664485 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c55f8d-6337-4bc9-9c92-17bbc02533e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:17.665306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c55f8d-6337-4bc9-9c92-17bbc02533e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:17.668710 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c55f8d-6337-4bc9-9c92-17bbc02533e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:17.673893 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c55f8d-6337-4bc9-9c92-17bbc02533e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:17.676204 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c55f8d-6337-4bc9-9c92-17bbc02533e4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:17.792576 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1018/2036] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:26:17 2022] POST /identity/v3/projects => generated 399 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:26:17.795445 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae63b9a-0c52-41ee-9b38-be7848c657dc admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:17.838598 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae63b9a-0c52-41ee-9b38-be7848c657dc admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:18.058878 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e194d5-1ff7-4a3d-9527-ea72afc2a98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:18.060703 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e194d5-1ff7-4a3d-9527-ea72afc2a98d admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:18.061345 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e194d5-1ff7-4a3d-9527-ea72afc2a98d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:18.062044 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e194d5-1ff7-4a3d-9527-ea72afc2a98d admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:18.063637 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e194d5-1ff7-4a3d-9527-ea72afc2a98d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:18.074611 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e194d5-1ff7-4a3d-9527-ea72afc2a98d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:18.136301 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1019/2037] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:26:17 2022] POST /identity/v3/users => generated 300 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:26:18.141184 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c55f8d-6337-4bc9-9c92-17bbc02533e4 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:18.163087 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c55f8d-6337-4bc9-9c92-17bbc02533e4 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:18.179359 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b7adf-4575-41f7-ac53-70bf968f0911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:18.180717 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b7adf-4575-41f7-ac53-70bf968f0911 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:18.181366 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b7adf-4575-41f7-ac53-70bf968f0911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:18.181940 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b7adf-4575-41f7-ac53-70bf968f0911 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:18.184045 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b7adf-4575-41f7-ac53-70bf968f0911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:18.187644 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b7adf-4575-41f7-ac53-70bf968f0911 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:18.218742 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1019/2038] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:26:18 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:26:18.224438 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e194d5-1ff7-4a3d-9527-ea72afc2a98d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:18.251007 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e194d5-1ff7-4a3d-9527-ea72afc2a98d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:18.277741 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d9dcd-6262-4ca0-98db-3200cfce8764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:18.279360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d9dcd-6262-4ca0-98db-3200cfce8764 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:18.279962 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d9dcd-6262-4ca0-98db-3200cfce8764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:18.280527 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d9dcd-6262-4ca0-98db-3200cfce8764 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:18.286810 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205d9dcd-6262-4ca0-98db-3200cfce8764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:18.290863 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205d9dcd-6262-4ca0-98db-3200cfce8764 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:18.333648 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1020/2039] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:26:18 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:26:18.342114 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b7adf-4575-41f7-ac53-70bf968f0911 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:18.381665 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b7adf-4575-41f7-ac53-70bf968f0911 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:18.399915 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a479d-ee14-448c-ab41-0bfab86ef02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:18.403169 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a479d-ee14-448c-ab41-0bfab86ef02e admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:18.403867 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a479d-ee14-448c-ab41-0bfab86ef02e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:18.404477 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a479d-ee14-448c-ab41-0bfab86ef02e admin admin] PATH_INFO: `/v3/projects/fbb825c2fe9e4dfb8ce71bfbd7eafecb/users/e7ad7c11acf647b086df23f7250a1a35/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:18.476085 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616a479d-ee14-448c-ab41-0bfab86ef02e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbb825c2fe9e4dfb8ce71bfbd7eafecb, user_id=e7ad7c11acf647b086df23f7250a1a35, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:18.504045 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616a479d-ee14-448c-ab41-0bfab86ef02e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:18.576928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1020/2040] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:26:18 2022] PUT /identity/v3/projects/fbb825c2fe9e4dfb8ce71bfbd7eafecb/users/e7ad7c11acf647b086df23f7250a1a35/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:26:18.590850 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072137ba-2397-4b2d-bac3-e4d30ec633e0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:18.591692 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072137ba-2397-4b2d-bac3-e4d30ec633e0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:18.592460 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072137ba-2397-4b2d-bac3-e4d30ec633e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:18.729268 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-072137ba-2397-4b2d-bac3-e4d30ec633e0 None None] MFA Rules not processed for user `e7ad7c11acf647b086df23f7250a1a35`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:18.732702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-072137ba-2397-4b2d-bac3-e4d30ec633e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:18.872534 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:19.017168 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1021/2041] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:18 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 428 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:26:19.345693 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a479d-ee14-448c-ab41-0bfab86ef02e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:19.351394 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-616a479d-ee14-448c-ab41-0bfab86ef02e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:19.494833 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a479d-ee14-448c-ab41-0bfab86ef02e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:19.516918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23525712-3bdf-4987-b224-032f416d12eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:19.520957 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23525712-3bdf-4987-b224-032f416d12eb service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:19.521617 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23525712-3bdf-4987-b224-032f416d12eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:19.522153 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23525712-3bdf-4987-b224-032f416d12eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:19.550538 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23525712-3bdf-4987-b224-032f416d12eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:19.553518 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23525712-3bdf-4987-b224-032f416d12eb service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:19.573132 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1021/2042] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:26:19 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:26:29.007107 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=15 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:29.566931 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=75,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:43.027072 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072137ba-2397-4b2d-bac3-e4d30ec633e0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:43.154060 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072137ba-2397-4b2d-bac3-e4d30ec633e0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:43.162618 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5047ab1a-131e-4d15-b245-fad4e7406846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:43.171970 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047ab1a-131e-4d15-b245-fad4e7406846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:43.171970 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047ab1a-131e-4d15-b245-fad4e7406846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:43.171970 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047ab1a-131e-4d15-b245-fad4e7406846 admin admin] PATH_INFO: `/v3/users/e7ad7c11acf647b086df23f7250a1a35` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:43.171970 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5047ab1a-131e-4d15-b245-fad4e7406846 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7ad7c11acf647b086df23f7250a1a35)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:43.171970 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5047ab1a-131e-4d15-b245-fad4e7406846 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:43.301082 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5047ab1a-131e-4d15-b245-fad4e7406846 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7ad7c11acf647b086df23f7250a1a35'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:43.321018 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5047ab1a-131e-4d15-b245-fad4e7406846 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7ad7c11acf647b086df23f7250a1a35'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:43.341982 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5047ab1a-131e-4d15-b245-fad4e7406846 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7ad7c11acf647b086df23f7250a1a35'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:43.341982 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5047ab1a-131e-4d15-b245-fad4e7406846 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7ad7c11acf647b086df23f7250a1a35'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:43.341982 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5047ab1a-131e-4d15-b245-fad4e7406846 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7ad7c11acf647b086df23f7250a1a35'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:43.365309 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1022/2043] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:26:43 2022] DELETE /identity/v3/users/e7ad7c11acf647b086df23f7250a1a35 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:26:43.373187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a328c03-a668-4c43-8a52-4446cd864e12 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:43.374073 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a328c03-a668-4c43-8a52-4446cd864e12 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:43.374073 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a328c03-a668-4c43-8a52-4446cd864e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:43.405622 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5a328c03-a668-4c43-8a52-4446cd864e12 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:43.407654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5a328c03-a668-4c43-8a52-4446cd864e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:43.510409 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1022/2044] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:26:43.528640 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5047ab1a-131e-4d15-b245-fad4e7406846 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:43.531996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5047ab1a-131e-4d15-b245-fad4e7406846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:43.644640 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5047ab1a-131e-4d15-b245-fad4e7406846 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:43.650849 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818b036-6e59-44d3-90a9-7c72842eeff7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:43.651633 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818b036-6e59-44d3-90a9-7c72842eeff7 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:43.651980 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818b036-6e59-44d3-90a9-7c72842eeff7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:43.652326 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818b036-6e59-44d3-90a9-7c72842eeff7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:43.669819 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6818b036-6e59-44d3-90a9-7c72842eeff7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:43.671345 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6818b036-6e59-44d3-90a9-7c72842eeff7 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:43.685668 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1023/2045] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:26:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:26:44.233044 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a328c03-a668-4c43-8a52-4446cd864e12 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:44.235581 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5a328c03-a668-4c43-8a52-4446cd864e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:44.273759 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a328c03-a668-4c43-8a52-4446cd864e12 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:44.287454 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f41232-6c5b-4ed1-8c85-92f2a8c62aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:44.289741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f41232-6c5b-4ed1-8c85-92f2a8c62aef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:44.290260 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f41232-6c5b-4ed1-8c85-92f2a8c62aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:44.290883 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f41232-6c5b-4ed1-8c85-92f2a8c62aef admin admin] PATH_INFO: `/v3/projects/fbb825c2fe9e4dfb8ce71bfbd7eafecb` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:44.292932 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f41232-6c5b-4ed1-8c85-92f2a8c62aef admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbb825c2fe9e4dfb8ce71bfbd7eafecb)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:44.296918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f41232-6c5b-4ed1-8c85-92f2a8c62aef admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:44.390416 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-44f41232-6c5b-4ed1-8c85-92f2a8c62aef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbb825c2fe9e4dfb8ce71bfbd7eafecb', 'request_id': 'req-44f41232-6c5b-4ed1-8c85-92f2a8c62aef'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:44.403893 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-44f41232-6c5b-4ed1-8c85-92f2a8c62aef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbb825c2fe9e4dfb8ce71bfbd7eafecb', 'request_id': 'req-44f41232-6c5b-4ed1-8c85-92f2a8c62aef'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:44.416217 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-44f41232-6c5b-4ed1-8c85-92f2a8c62aef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbb825c2fe9e4dfb8ce71bfbd7eafecb', 'request_id': 'req-44f41232-6c5b-4ed1-8c85-92f2a8c62aef'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:44.459212 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-44f41232-6c5b-4ed1-8c85-92f2a8c62aef admin admin] The token cache is being invalidate because project fbb825c2fe9e4dfb8ce71bfbd7eafecb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:26:44.459212 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-44f41232-6c5b-4ed1-8c85-92f2a8c62aef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:44.467806 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-44f41232-6c5b-4ed1-8c85-92f2a8c62aef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:44.479574 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1023/2046] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:26:44 2022] DELETE /identity/v3/projects/fbb825c2fe9e4dfb8ce71bfbd7eafecb => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:26:44.642939 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818b036-6e59-44d3-90a9-7c72842eeff7 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:44.649348 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6818b036-6e59-44d3-90a9-7c72842eeff7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:44.685025 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818b036-6e59-44d3-90a9-7c72842eeff7 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:44.732503 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:44.830569 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14d9f9a-f001-4108-be9b-5b0153ccdae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:44.832156 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d9f9a-f001-4108-be9b-5b0153ccdae3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:44.832900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d9f9a-f001-4108-be9b-5b0153ccdae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:44.833624 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d9f9a-f001-4108-be9b-5b0153ccdae3 admin admin] PATH_INFO: `/v3/users/1c19857e7eee4fffa63f6ba507e119cf` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:44.836188 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14d9f9a-f001-4108-be9b-5b0153ccdae3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c19857e7eee4fffa63f6ba507e119cf)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:44.839896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14d9f9a-f001-4108-be9b-5b0153ccdae3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:44.952812 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a14d9f9a-f001-4108-be9b-5b0153ccdae3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c19857e7eee4fffa63f6ba507e119cf'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:44.971182 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a14d9f9a-f001-4108-be9b-5b0153ccdae3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c19857e7eee4fffa63f6ba507e119cf'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:44.987499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a14d9f9a-f001-4108-be9b-5b0153ccdae3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c19857e7eee4fffa63f6ba507e119cf'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:44.988705 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a14d9f9a-f001-4108-be9b-5b0153ccdae3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c19857e7eee4fffa63f6ba507e119cf'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:44.989425 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a14d9f9a-f001-4108-be9b-5b0153ccdae3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c19857e7eee4fffa63f6ba507e119cf'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:45.019588 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1024/2047] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:26:44 2022] DELETE /identity/v3/users/1c19857e7eee4fffa63f6ba507e119cf => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:26:45.039899 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cdf56f-a59b-4fa4-80c1-9303179a03b5 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:45.039899 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cdf56f-a59b-4fa4-80c1-9303179a03b5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:45.039899 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cdf56f-a59b-4fa4-80c1-9303179a03b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:45.063943 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-29cdf56f-a59b-4fa4-80c1-9303179a03b5 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:45.066585 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-29cdf56f-a59b-4fa4-80c1-9303179a03b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:45.154880 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1024/2048] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:26:45.171057 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14d9f9a-f001-4108-be9b-5b0153ccdae3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:45.173172 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a14d9f9a-f001-4108-be9b-5b0153ccdae3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:45.275535 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14d9f9a-f001-4108-be9b-5b0153ccdae3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:45.280635 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2acb06-7578-4f81-8d71-08f8fa3c3772 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:45.281450 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2acb06-7578-4f81-8d71-08f8fa3c3772 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:45.281839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2acb06-7578-4f81-8d71-08f8fa3c3772 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:45.282207 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2acb06-7578-4f81-8d71-08f8fa3c3772 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:45.299864 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2acb06-7578-4f81-8d71-08f8fa3c3772 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:45.299864 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2acb06-7578-4f81-8d71-08f8fa3c3772 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:45.318501 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1025/2049] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:26:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:26:45.993860 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cdf56f-a59b-4fa4-80c1-9303179a03b5 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:45.998200 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-29cdf56f-a59b-4fa4-80c1-9303179a03b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:46.031130 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cdf56f-a59b-4fa4-80c1-9303179a03b5 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:46.045625 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195f98b-3c6f-44a6-9fc1-3490f924fc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:46.047819 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195f98b-3c6f-44a6-9fc1-3490f924fc67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:46.048361 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195f98b-3c6f-44a6-9fc1-3490f924fc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:46.048883 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195f98b-3c6f-44a6-9fc1-3490f924fc67 admin admin] PATH_INFO: `/v3/projects/208f76a478cf4467a0a41f26ef57bf7b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:46.052470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5195f98b-3c6f-44a6-9fc1-3490f924fc67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=208f76a478cf4467a0a41f26ef57bf7b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:46.055349 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5195f98b-3c6f-44a6-9fc1-3490f924fc67 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:46.096697 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:46.164012 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5195f98b-3c6f-44a6-9fc1-3490f924fc67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '208f76a478cf4467a0a41f26ef57bf7b', 'request_id': 'req-5195f98b-3c6f-44a6-9fc1-3490f924fc67'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:46.184259 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5195f98b-3c6f-44a6-9fc1-3490f924fc67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '208f76a478cf4467a0a41f26ef57bf7b', 'request_id': 'req-5195f98b-3c6f-44a6-9fc1-3490f924fc67'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:46.195797 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5195f98b-3c6f-44a6-9fc1-3490f924fc67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '208f76a478cf4467a0a41f26ef57bf7b', 'request_id': 'req-5195f98b-3c6f-44a6-9fc1-3490f924fc67'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:46.229875 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5195f98b-3c6f-44a6-9fc1-3490f924fc67 admin admin] The token cache is being invalidate because project 208f76a478cf4467a0a41f26ef57bf7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:26:46.230337 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5195f98b-3c6f-44a6-9fc1-3490f924fc67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:46.231587 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5195f98b-3c6f-44a6-9fc1-3490f924fc67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:46.238128 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1025/2050] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:26:45 2022] DELETE /identity/v3/projects/208f76a478cf4467a0a41f26ef57bf7b => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:26:46.239943 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2acb06-7578-4f81-8d71-08f8fa3c3772 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:46.255484 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2acb06-7578-4f81-8d71-08f8fa3c3772 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:46.363773 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:46.398091 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51f6b59-bcca-41d3-bbe6-446d7d0936fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:46.399724 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f6b59-bcca-41d3-bbe6-446d7d0936fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:46.400249 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f6b59-bcca-41d3-bbe6-446d7d0936fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:46.400782 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f6b59-bcca-41d3-bbe6-446d7d0936fd admin admin] PATH_INFO: `/v3/users/ee1241fbe2f448a6a7fc5969e4b0266e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:46.404652 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51f6b59-bcca-41d3-bbe6-446d7d0936fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee1241fbe2f448a6a7fc5969e4b0266e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:46.406012 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51f6b59-bcca-41d3-bbe6-446d7d0936fd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:46.518754 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a51f6b59-bcca-41d3-bbe6-446d7d0936fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee1241fbe2f448a6a7fc5969e4b0266e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:46.537568 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a51f6b59-bcca-41d3-bbe6-446d7d0936fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee1241fbe2f448a6a7fc5969e4b0266e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:46.549867 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a51f6b59-bcca-41d3-bbe6-446d7d0936fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee1241fbe2f448a6a7fc5969e4b0266e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:46.550854 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a51f6b59-bcca-41d3-bbe6-446d7d0936fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee1241fbe2f448a6a7fc5969e4b0266e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:46.551781 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a51f6b59-bcca-41d3-bbe6-446d7d0936fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee1241fbe2f448a6a7fc5969e4b0266e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:46.583157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1026/2051] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:26:46 2022] DELETE /identity/v3/users/ee1241fbe2f448a6a7fc5969e4b0266e => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:26:47.201601 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195f98b-3c6f-44a6-9fc1-3490f924fc67 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:47.203766 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5195f98b-3c6f-44a6-9fc1-3490f924fc67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:47.220669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195f98b-3c6f-44a6-9fc1-3490f924fc67 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:47.338974 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6856d5-abb8-4146-90a0-c64f31553abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:47.340824 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6856d5-abb8-4146-90a0-c64f31553abe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:47.341350 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6856d5-abb8-4146-90a0-c64f31553abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:47.341862 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6856d5-abb8-4146-90a0-c64f31553abe admin admin] PATH_INFO: `/v3/projects/ab9fc4f5e92a4f53a1bcbc00928e3784` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:47.344302 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6856d5-abb8-4146-90a0-c64f31553abe admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab9fc4f5e92a4f53a1bcbc00928e3784)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:47.348381 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6856d5-abb8-4146-90a0-c64f31553abe admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:47.442184 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fb6856d5-abb8-4146-90a0-c64f31553abe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab9fc4f5e92a4f53a1bcbc00928e3784', 'request_id': 'req-fb6856d5-abb8-4146-90a0-c64f31553abe'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:47.455770 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fb6856d5-abb8-4146-90a0-c64f31553abe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab9fc4f5e92a4f53a1bcbc00928e3784', 'request_id': 'req-fb6856d5-abb8-4146-90a0-c64f31553abe'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:47.465600 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fb6856d5-abb8-4146-90a0-c64f31553abe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab9fc4f5e92a4f53a1bcbc00928e3784', 'request_id': 'req-fb6856d5-abb8-4146-90a0-c64f31553abe'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:47.485476 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fb6856d5-abb8-4146-90a0-c64f31553abe admin admin] The token cache is being invalidate because project ab9fc4f5e92a4f53a1bcbc00928e3784 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:26:47.485986 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fb6856d5-abb8-4146-90a0-c64f31553abe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:47.487164 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fb6856d5-abb8-4146-90a0-c64f31553abe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:47.490307 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1026/2052] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:26:47 2022] DELETE /identity/v3/projects/ab9fc4f5e92a4f53a1bcbc00928e3784 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:26:47.499062 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c9cb0-68c3-4879-a04c-fde1d48bb8c4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:47.499971 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c9cb0-68c3-4879-a04c-fde1d48bb8c4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:47.500147 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c9cb0-68c3-4879-a04c-fde1d48bb8c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:47.537724 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-775c9cb0-68c3-4879-a04c-fde1d48bb8c4 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:47.540601 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-775c9cb0-68c3-4879-a04c-fde1d48bb8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:47.666535 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1027/2053] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:26:47.681526 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4961fa2b-f66b-49b0-9e9d-baf2639fff4f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:47.682013 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4961fa2b-f66b-49b0-9e9d-baf2639fff4f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:47.682500 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4961fa2b-f66b-49b0-9e9d-baf2639fff4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:47.712220 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4961fa2b-f66b-49b0-9e9d-baf2639fff4f None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:47.713848 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4961fa2b-f66b-49b0-9e9d-baf2639fff4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:47.723773 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1027/2054] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:26:47.730157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c9cb0-68c3-4879-a04c-fde1d48bb8c4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:47.749749 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c9cb0-68c3-4879-a04c-fde1d48bb8c4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:47.758496 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a3856-3ebc-4d79-939d-7bd873fdc9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:47.759676 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a3856-3ebc-4d79-939d-7bd873fdc9ad admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:47.760145 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a3856-3ebc-4d79-939d-7bd873fdc9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:47.760643 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a3856-3ebc-4d79-939d-7bd873fdc9ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:47.761976 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a3856-3ebc-4d79-939d-7bd873fdc9ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:26:47.762524 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a3856-3ebc-4d79-939d-7bd873fdc9ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:47.765574 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a3856-3ebc-4d79-939d-7bd873fdc9ad admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:47.787439 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1028/2055] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:26:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:26:47.795662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4961fa2b-f66b-49b0-9e9d-baf2639fff4f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:47.814713 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4961fa2b-f66b-49b0-9e9d-baf2639fff4f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:47.824554 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5578c3-29af-486a-9328-59c45e85fbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:47.825741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5578c3-29af-486a-9328-59c45e85fbd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:47.826418 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5578c3-29af-486a-9328-59c45e85fbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:47.827008 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5578c3-29af-486a-9328-59c45e85fbd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:47.828411 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5578c3-29af-486a-9328-59c45e85fbd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:47.832507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5578c3-29af-486a-9328-59c45e85fbd9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:47.884720 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1028/2056] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:26:47 2022] POST /identity/v3/projects => generated 367 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:26:47.889880 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a3856-3ebc-4d79-939d-7bd873fdc9ad admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:47.904080 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a3856-3ebc-4d79-939d-7bd873fdc9ad admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:47.976092 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:48.002464 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63248678-9009-4ee9-b820-5dc31541ca96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:48.003863 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63248678-9009-4ee9-b820-5dc31541ca96 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:48.004452 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63248678-9009-4ee9-b820-5dc31541ca96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:48.005015 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63248678-9009-4ee9-b820-5dc31541ca96 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:48.007661 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63248678-9009-4ee9-b820-5dc31541ca96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:48.011584 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63248678-9009-4ee9-b820-5dc31541ca96 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:48.058912 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1029/2057] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:26:47 2022] POST /identity/v3/users => generated 284 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:26:48.065531 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5578c3-29af-486a-9328-59c45e85fbd9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:48.086964 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5578c3-29af-486a-9328-59c45e85fbd9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:48.097417 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e048516-20c9-46f1-87e3-a98450251d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:48.098646 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e048516-20c9-46f1-87e3-a98450251d75 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:48.099171 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e048516-20c9-46f1-87e3-a98450251d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:48.099722 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e048516-20c9-46f1-87e3-a98450251d75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:48.101342 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e048516-20c9-46f1-87e3-a98450251d75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:48.104710 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e048516-20c9-46f1-87e3-a98450251d75 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:48.109092 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:48.129312 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1029/2058] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:26:48 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:26:48.134474 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63248678-9009-4ee9-b820-5dc31541ca96 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:48.146583 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63248678-9009-4ee9-b820-5dc31541ca96 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:48.153060 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5580078-08ff-4c8d-a9e5-94d4e3c964dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:48.153911 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5580078-08ff-4c8d-a9e5-94d4e3c964dc admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:48.154206 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5580078-08ff-4c8d-a9e5-94d4e3c964dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:48.154499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5580078-08ff-4c8d-a9e5-94d4e3c964dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:48.155269 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5580078-08ff-4c8d-a9e5-94d4e3c964dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:48.157009 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5580078-08ff-4c8d-a9e5-94d4e3c964dc admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:48.180825 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1030/2059] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:26:48 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:26:48.188296 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e048516-20c9-46f1-87e3-a98450251d75 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:48.204847 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e048516-20c9-46f1-87e3-a98450251d75 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:48.214756 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de4675-35a3-4b12-be7f-26c3e493c9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:48.215913 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de4675-35a3-4b12-be7f-26c3e493c9ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:48.216479 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de4675-35a3-4b12-be7f-26c3e493c9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:48.216883 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de4675-35a3-4b12-be7f-26c3e493c9ae admin admin] PATH_INFO: `/v3/projects/d708923adb624509b7c2e462475b410e/users/243821bfd5e34fcfa649dd92ab7397f6/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:48.243310 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31de4675-35a3-4b12-be7f-26c3e493c9ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=d708923adb624509b7c2e462475b410e, user_id=243821bfd5e34fcfa649dd92ab7397f6, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:48.248618 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31de4675-35a3-4b12-be7f-26c3e493c9ae admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:48.283919 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1030/2060] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:26:48 2022] PUT /identity/v3/projects/d708923adb624509b7c2e462475b410e/users/243821bfd5e34fcfa649dd92ab7397f6/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:26:48.303187 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71d840-aab1-401e-b696-f4946ec6322f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:48.303187 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71d840-aab1-401e-b696-f4946ec6322f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:48.303701 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71d840-aab1-401e-b696-f4946ec6322f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:48.355946 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-fe71d840-aab1-401e-b696-f4946ec6322f None None] MFA Rules not processed for user `243821bfd5e34fcfa649dd92ab7397f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:48.358667 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fe71d840-aab1-401e-b696-f4946ec6322f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:48.502591 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1031/2061] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:48 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:26:48.510756 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de4675-35a3-4b12-be7f-26c3e493c9ae admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:48.623050 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de4675-35a3-4b12-be7f-26c3e493c9ae admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:48.632670 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b25e16-59d4-4126-a7b6-111689c0ad5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:48.633621 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b25e16-59d4-4126-a7b6-111689c0ad5c admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:48.633904 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b25e16-59d4-4126-a7b6-111689c0ad5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:48.634270 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b25e16-59d4-4126-a7b6-111689c0ad5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:48.635119 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b25e16-59d4-4126-a7b6-111689c0ad5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:48.637982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b25e16-59d4-4126-a7b6-111689c0ad5c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:48.692181 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1031/2062] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:26:48 2022] POST /identity/v3/projects => generated 367 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:26:48.699707 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71d840-aab1-401e-b696-f4946ec6322f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:48.720015 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71d840-aab1-401e-b696-f4946ec6322f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:48.843607 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca489ca3-90a2-405b-a2d8-beb4e1d7a81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:48.844897 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca489ca3-90a2-405b-a2d8-beb4e1d7a81e admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:48.845423 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca489ca3-90a2-405b-a2d8-beb4e1d7a81e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:48.845935 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca489ca3-90a2-405b-a2d8-beb4e1d7a81e admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:48.847266 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca489ca3-90a2-405b-a2d8-beb4e1d7a81e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:48.851472 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca489ca3-90a2-405b-a2d8-beb4e1d7a81e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:48.892737 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1032/2063] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:26:48 2022] POST /identity/v3/users => generated 290 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:26:48.897851 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b25e16-59d4-4126-a7b6-111689c0ad5c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:48.911109 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b25e16-59d4-4126-a7b6-111689c0ad5c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:48.921219 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b628f9db-e7c3-4cdb-8d9c-137238b535bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:48.924800 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628f9db-e7c3-4cdb-8d9c-137238b535bd admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:48.925774 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628f9db-e7c3-4cdb-8d9c-137238b535bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:48.926826 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628f9db-e7c3-4cdb-8d9c-137238b535bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:48.928389 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b628f9db-e7c3-4cdb-8d9c-137238b535bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:48.933388 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b628f9db-e7c3-4cdb-8d9c-137238b535bd admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:48.961956 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1032/2064] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:26:48 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:26:48.969129 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca489ca3-90a2-405b-a2d8-beb4e1d7a81e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:48.985483 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca489ca3-90a2-405b-a2d8-beb4e1d7a81e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:48.994189 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab4f09-8f89-4a46-8036-740ac88a798f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:48.995475 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab4f09-8f89-4a46-8036-740ac88a798f admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:48.996034 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab4f09-8f89-4a46-8036-740ac88a798f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:48.996531 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab4f09-8f89-4a46-8036-740ac88a798f admin admin] PATH_INFO: `/v3/projects/a2d36c976c054628bfe77801d7d385f6/users/bdaa64d47c1c4ada82ffcbedb6181e72/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:49.021896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eab4f09-8f89-4a46-8036-740ac88a798f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2d36c976c054628bfe77801d7d385f6, user_id=bdaa64d47c1c4ada82ffcbedb6181e72, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:49.026117 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eab4f09-8f89-4a46-8036-740ac88a798f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:49.055110 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1033/2065] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:26:48 2022] PUT /identity/v3/projects/a2d36c976c054628bfe77801d7d385f6/users/bdaa64d47c1c4ada82ffcbedb6181e72/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:26:49.057318 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b628f9db-e7c3-4cdb-8d9c-137238b535bd admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:49.070628 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b628f9db-e7c3-4cdb-8d9c-137238b535bd admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:49.185858 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bca1e9-1c7a-42ef-9c43-4c3a1cea0b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:49.186638 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bca1e9-1c7a-42ef-9c43-4c3a1cea0b76 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:49.187061 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bca1e9-1c7a-42ef-9c43-4c3a1cea0b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:49.187347 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bca1e9-1c7a-42ef-9c43-4c3a1cea0b76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:49.188408 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bca1e9-1c7a-42ef-9c43-4c3a1cea0b76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:49.191360 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bca1e9-1c7a-42ef-9c43-4c3a1cea0b76 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:49.208414 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1033/2066] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:26:49.214878 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab4f09-8f89-4a46-8036-740ac88a798f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:49.231760 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab4f09-8f89-4a46-8036-740ac88a798f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:49.236970 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faef97e9-79fb-42af-a635-e18a0ea11a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:49.237889 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef97e9-79fb-42af-a635-e18a0ea11a22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:49.238176 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef97e9-79fb-42af-a635-e18a0ea11a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:49.238447 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faef97e9-79fb-42af-a635-e18a0ea11a22 admin admin] PATH_INFO: `/v3/projects/a2d36c976c054628bfe77801d7d385f6/users/bdaa64d47c1c4ada82ffcbedb6181e72/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:49.240698 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faef97e9-79fb-42af-a635-e18a0ea11a22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2d36c976c054628bfe77801d7d385f6, user_id=bdaa64d47c1c4ada82ffcbedb6181e72, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:49.243765 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faef97e9-79fb-42af-a635-e18a0ea11a22 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:49.265101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1034/2067] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:26:49 2022] PUT /identity/v3/projects/a2d36c976c054628bfe77801d7d385f6/users/bdaa64d47c1c4ada82ffcbedb6181e72/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:26:49.285426 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8f4f1-515c-4afa-b2b9-69a613bd4db8 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:49.285998 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8f4f1-515c-4afa-b2b9-69a613bd4db8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:49.286540 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8f4f1-515c-4afa-b2b9-69a613bd4db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:49.328730 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b9b8f4f1-515c-4afa-b2b9-69a613bd4db8 None None] MFA Rules not processed for user `bdaa64d47c1c4ada82ffcbedb6181e72`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:49.329733 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b9b8f4f1-515c-4afa-b2b9-69a613bd4db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:49.383733 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:49.440075 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1034/2068] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:49 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:26:49.458649 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faef97e9-79fb-42af-a635-e18a0ea11a22 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:49.462364 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-faef97e9-79fb-42af-a635-e18a0ea11a22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:49.485406 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:49.578447 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faef97e9-79fb-42af-a635-e18a0ea11a22 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:49.585176 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c42e6-6bda-421c-9ec0-d69e820f1637 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:49.586196 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c42e6-6bda-421c-9ec0-d69e820f1637 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:49.586839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c42e6-6bda-421c-9ec0-d69e820f1637 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:49.587311 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c42e6-6bda-421c-9ec0-d69e820f1637 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:49.604599 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c42e6-6bda-421c-9ec0-d69e820f1637 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:49.607751 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c42e6-6bda-421c-9ec0-d69e820f1637 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:49.619028 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1035/2069] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:26:49 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:26:52.294682 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b8f4f1-515c-4afa-b2b9-69a613bd4db8 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:52.409779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b8f4f1-515c-4afa-b2b9-69a613bd4db8 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:52.421128 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de4d971-6fcd-4a31-bc89-07fef4422920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:52.423095 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de4d971-6fcd-4a31-bc89-07fef4422920 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:52.423719 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de4d971-6fcd-4a31-bc89-07fef4422920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:52.424227 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de4d971-6fcd-4a31-bc89-07fef4422920 admin admin] PATH_INFO: `/v3/users/243821bfd5e34fcfa649dd92ab7397f6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:52.426671 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de4d971-6fcd-4a31-bc89-07fef4422920 admin admin] RBAC: Authorizing `identity:delete_user(user_id=243821bfd5e34fcfa649dd92ab7397f6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:52.432538 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de4d971-6fcd-4a31-bc89-07fef4422920 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:52.524705 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2de4d971-6fcd-4a31-bc89-07fef4422920 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '243821bfd5e34fcfa649dd92ab7397f6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:52.543909 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2de4d971-6fcd-4a31-bc89-07fef4422920 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '243821bfd5e34fcfa649dd92ab7397f6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:52.563124 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2de4d971-6fcd-4a31-bc89-07fef4422920 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '243821bfd5e34fcfa649dd92ab7397f6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:52.564645 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2de4d971-6fcd-4a31-bc89-07fef4422920 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '243821bfd5e34fcfa649dd92ab7397f6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:52.601396 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2de4d971-6fcd-4a31-bc89-07fef4422920 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '243821bfd5e34fcfa649dd92ab7397f6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:52.614589 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1035/2070] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:26:52 2022] DELETE /identity/v3/users/243821bfd5e34fcfa649dd92ab7397f6 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:26:52.621056 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c3903-3b0b-4ee3-b65b-b29421a510b8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:52.621801 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c3903-3b0b-4ee3-b65b-b29421a510b8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:52.621801 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c3903-3b0b-4ee3-b65b-b29421a510b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:52.656480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-462c3903-3b0b-4ee3-b65b-b29421a510b8 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:52.659187 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-462c3903-3b0b-4ee3-b65b-b29421a510b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:52.788674 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1036/2071] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:26:52.804751 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de4d971-6fcd-4a31-bc89-07fef4422920 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:52.806628 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2de4d971-6fcd-4a31-bc89-07fef4422920 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:52.916918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:52.924925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de4d971-6fcd-4a31-bc89-07fef4422920 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:52.930961 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb1889-8b39-4581-a8f5-ba6126446081 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:52.931756 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb1889-8b39-4581-a8f5-ba6126446081 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:52.932106 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb1889-8b39-4581-a8f5-ba6126446081 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:52.932460 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb1889-8b39-4581-a8f5-ba6126446081 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:52.944983 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eb1889-8b39-4581-a8f5-ba6126446081 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:52.946629 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eb1889-8b39-4581-a8f5-ba6126446081 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:52.962138 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1036/2072] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:26:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:26:53.361701 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c3903-3b0b-4ee3-b65b-b29421a510b8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:53.364180 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-462c3903-3b0b-4ee3-b65b-b29421a510b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:53.386650 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c3903-3b0b-4ee3-b65b-b29421a510b8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:53.401742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e354fb1-529f-4487-8bb5-2994e607eb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:53.404148 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e354fb1-529f-4487-8bb5-2994e607eb7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:53.406378 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e354fb1-529f-4487-8bb5-2994e607eb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:53.407397 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e354fb1-529f-4487-8bb5-2994e607eb7b admin admin] PATH_INFO: `/v3/projects/d708923adb624509b7c2e462475b410e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:53.409602 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e354fb1-529f-4487-8bb5-2994e607eb7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d708923adb624509b7c2e462475b410e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:53.413922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e354fb1-529f-4487-8bb5-2994e607eb7b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:53.509926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0e354fb1-529f-4487-8bb5-2994e607eb7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd708923adb624509b7c2e462475b410e', 'request_id': 'req-0e354fb1-529f-4487-8bb5-2994e607eb7b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:53.522456 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0e354fb1-529f-4487-8bb5-2994e607eb7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd708923adb624509b7c2e462475b410e', 'request_id': 'req-0e354fb1-529f-4487-8bb5-2994e607eb7b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:53.535170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0e354fb1-529f-4487-8bb5-2994e607eb7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd708923adb624509b7c2e462475b410e', 'request_id': 'req-0e354fb1-529f-4487-8bb5-2994e607eb7b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:53.559429 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0e354fb1-529f-4487-8bb5-2994e607eb7b admin admin] The token cache is being invalidate because project d708923adb624509b7c2e462475b410e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:26:53.560137 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0e354fb1-529f-4487-8bb5-2994e607eb7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:53.561457 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0e354fb1-529f-4487-8bb5-2994e607eb7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:53.564304 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1037/2073] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:26:53 2022] DELETE /identity/v3/projects/d708923adb624509b7c2e462475b410e => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:26:53.571735 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb1889-8b39-4581-a8f5-ba6126446081 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:53.590896 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb1889-8b39-4581-a8f5-ba6126446081 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:53.700433 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05755d-b079-4f0c-8c3a-b7886bc7152e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:53.701890 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05755d-b079-4f0c-8c3a-b7886bc7152e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:53.702415 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05755d-b079-4f0c-8c3a-b7886bc7152e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:53.702858 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05755d-b079-4f0c-8c3a-b7886bc7152e admin admin] PATH_INFO: `/v3/users/bdaa64d47c1c4ada82ffcbedb6181e72` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:53.705073 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea05755d-b079-4f0c-8c3a-b7886bc7152e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdaa64d47c1c4ada82ffcbedb6181e72)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:53.708667 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea05755d-b079-4f0c-8c3a-b7886bc7152e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:53.801420 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ea05755d-b079-4f0c-8c3a-b7886bc7152e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdaa64d47c1c4ada82ffcbedb6181e72'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:53.816810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ea05755d-b079-4f0c-8c3a-b7886bc7152e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdaa64d47c1c4ada82ffcbedb6181e72'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:53.832868 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ea05755d-b079-4f0c-8c3a-b7886bc7152e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdaa64d47c1c4ada82ffcbedb6181e72'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:53.834039 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ea05755d-b079-4f0c-8c3a-b7886bc7152e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdaa64d47c1c4ada82ffcbedb6181e72'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:53.862341 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ea05755d-b079-4f0c-8c3a-b7886bc7152e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdaa64d47c1c4ada82ffcbedb6181e72'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:53.875111 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1037/2074] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:26:53 2022] DELETE /identity/v3/users/bdaa64d47c1c4ada82ffcbedb6181e72 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:26:54.424738 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e354fb1-529f-4487-8bb5-2994e607eb7b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:54.428849 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0e354fb1-529f-4487-8bb5-2994e607eb7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:54.464592 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e354fb1-529f-4487-8bb5-2994e607eb7b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:54.550104 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:54.603180 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b120b10a-2aa6-4066-9d9a-da96c70f7c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:54.605050 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120b10a-2aa6-4066-9d9a-da96c70f7c6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:54.606131 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120b10a-2aa6-4066-9d9a-da96c70f7c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:54.607051 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120b10a-2aa6-4066-9d9a-da96c70f7c6b admin admin] PATH_INFO: `/v3/projects/a2d36c976c054628bfe77801d7d385f6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:54.610279 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b120b10a-2aa6-4066-9d9a-da96c70f7c6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2d36c976c054628bfe77801d7d385f6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:54.615030 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b120b10a-2aa6-4066-9d9a-da96c70f7c6b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:54.729342 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b120b10a-2aa6-4066-9d9a-da96c70f7c6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2d36c976c054628bfe77801d7d385f6', 'request_id': 'req-b120b10a-2aa6-4066-9d9a-da96c70f7c6b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:54.740347 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b120b10a-2aa6-4066-9d9a-da96c70f7c6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2d36c976c054628bfe77801d7d385f6', 'request_id': 'req-b120b10a-2aa6-4066-9d9a-da96c70f7c6b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:54.749363 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b120b10a-2aa6-4066-9d9a-da96c70f7c6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2d36c976c054628bfe77801d7d385f6', 'request_id': 'req-b120b10a-2aa6-4066-9d9a-da96c70f7c6b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:54.779472 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b120b10a-2aa6-4066-9d9a-da96c70f7c6b admin admin] The token cache is being invalidate because project a2d36c976c054628bfe77801d7d385f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:26:54.779817 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b120b10a-2aa6-4066-9d9a-da96c70f7c6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:54.780534 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b120b10a-2aa6-4066-9d9a-da96c70f7c6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:54.782161 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1038/2075] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:26:54 2022] DELETE /identity/v3/projects/a2d36c976c054628bfe77801d7d385f6 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:26:54.791926 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e3e9c-b62f-4dd4-b4ae-f25b876440df None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:54.793798 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e3e9c-b62f-4dd4-b4ae-f25b876440df None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:54.794913 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e3e9c-b62f-4dd4-b4ae-f25b876440df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:54.828053 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-668e3e9c-b62f-4dd4-b4ae-f25b876440df None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:54.829377 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-668e3e9c-b62f-4dd4-b4ae-f25b876440df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:54.972197 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1038/2076] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:26:54.984140 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43986f30-6f2f-4ddd-b0db-f725bf0d256a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:54.984718 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43986f30-6f2f-4ddd-b0db-f725bf0d256a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:54.985198 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43986f30-6f2f-4ddd-b0db-f725bf0d256a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:55.018604 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-43986f30-6f2f-4ddd-b0db-f725bf0d256a None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:55.020220 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-43986f30-6f2f-4ddd-b0db-f725bf0d256a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:55.038799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1039/2077] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:26:55.043096 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e3e9c-b62f-4dd4-b4ae-f25b876440df None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:55.065825 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e3e9c-b62f-4dd4-b4ae-f25b876440df None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:55.076456 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bcac8f-c377-4144-993d-ac1a4c1c62f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:55.077968 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcac8f-c377-4144-993d-ac1a4c1c62f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:55.078553 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcac8f-c377-4144-993d-ac1a4c1c62f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:55.079397 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcac8f-c377-4144-993d-ac1a4c1c62f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:55.081113 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bcac8f-c377-4144-993d-ac1a4c1c62f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:26:55.081684 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bcac8f-c377-4144-993d-ac1a4c1c62f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:55.091647 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bcac8f-c377-4144-993d-ac1a4c1c62f4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:55.101579 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:55.113924 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1039/2078] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:26:55.120254 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43986f30-6f2f-4ddd-b0db-f725bf0d256a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:55.136950 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43986f30-6f2f-4ddd-b0db-f725bf0d256a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:55.146593 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0951a5-2270-48d7-b5d3-b3791928ffc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:55.148887 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0951a5-2270-48d7-b5d3-b3791928ffc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:55.149433 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0951a5-2270-48d7-b5d3-b3791928ffc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:55.149977 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0951a5-2270-48d7-b5d3-b3791928ffc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:55.152898 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0951a5-2270-48d7-b5d3-b3791928ffc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:55.156951 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0951a5-2270-48d7-b5d3-b3791928ffc2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:55.233861 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1040/2079] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:26:55 2022] POST /identity/v3/projects => generated 379 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:26:55.239154 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bcac8f-c377-4144-993d-ac1a4c1c62f4 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:55.254862 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bcac8f-c377-4144-993d-ac1a4c1c62f4 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:55.409817 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd121b-5000-4f58-b360-5b2b26d6fe2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:55.410685 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd121b-5000-4f58-b360-5b2b26d6fe2a admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:55.411103 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd121b-5000-4f58-b360-5b2b26d6fe2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:55.411494 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd121b-5000-4f58-b360-5b2b26d6fe2a admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:55.412550 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dd121b-5000-4f58-b360-5b2b26d6fe2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:55.416569 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dd121b-5000-4f58-b360-5b2b26d6fe2a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:55.454135 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1040/2080] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:26:55 2022] POST /identity/v3/users => generated 290 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:26:55.459291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0951a5-2270-48d7-b5d3-b3791928ffc2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:55.473553 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0951a5-2270-48d7-b5d3-b3791928ffc2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:55.483262 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd239ef6-21af-4592-b7fd-6444c11a9dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:55.484681 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd239ef6-21af-4592-b7fd-6444c11a9dd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:55.485265 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd239ef6-21af-4592-b7fd-6444c11a9dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:55.485856 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd239ef6-21af-4592-b7fd-6444c11a9dd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:55.487337 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd239ef6-21af-4592-b7fd-6444c11a9dd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:55.491244 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd239ef6-21af-4592-b7fd-6444c11a9dd9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:55.514047 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1041/2081] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:26:55.521459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd121b-5000-4f58-b360-5b2b26d6fe2a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:55.567175 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd121b-5000-4f58-b360-5b2b26d6fe2a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:55.581035 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c737d1-e8b1-43a8-8dc6-ac718f8a687b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:55.582081 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c737d1-e8b1-43a8-8dc6-ac718f8a687b admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:55.582579 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c737d1-e8b1-43a8-8dc6-ac718f8a687b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:55.583095 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c737d1-e8b1-43a8-8dc6-ac718f8a687b admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:55.584786 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c737d1-e8b1-43a8-8dc6-ac718f8a687b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:55.588059 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c737d1-e8b1-43a8-8dc6-ac718f8a687b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:55.612943 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1041/2082] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:26:55.619084 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd239ef6-21af-4592-b7fd-6444c11a9dd9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:55.639932 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd239ef6-21af-4592-b7fd-6444c11a9dd9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:55.651579 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac144c-4974-47cb-9e4c-d60148f2d00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:55.653292 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac144c-4974-47cb-9e4c-d60148f2d00c admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:55.654028 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac144c-4974-47cb-9e4c-d60148f2d00c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:55.654627 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac144c-4974-47cb-9e4c-d60148f2d00c admin admin] PATH_INFO: `/v3/projects/470eead9dee24f1fb0660c5bffd7ed28/users/f9197a67739a40a4be3c9b8a65b2b1d6/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:55.690340 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ac144c-4974-47cb-9e4c-d60148f2d00c admin admin] RBAC: Authorizing `identity:create_grant(project_id=470eead9dee24f1fb0660c5bffd7ed28, user_id=f9197a67739a40a4be3c9b8a65b2b1d6, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:55.698825 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ac144c-4974-47cb-9e4c-d60148f2d00c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:55.737623 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1042/2083] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:26:55 2022] PUT /identity/v3/projects/470eead9dee24f1fb0660c5bffd7ed28/users/f9197a67739a40a4be3c9b8a65b2b1d6/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:26:55.753733 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b34464-46af-4352-9593-598afd9104bf None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:55.754375 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b34464-46af-4352-9593-598afd9104bf None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:55.755255 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b34464-46af-4352-9593-598afd9104bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:55.815239 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f4b34464-46af-4352-9593-598afd9104bf None None] MFA Rules not processed for user `f9197a67739a40a4be3c9b8a65b2b1d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:55.817177 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f4b34464-46af-4352-9593-598afd9104bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:55.957425 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1042/2084] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:26:55.965413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac144c-4974-47cb-9e4c-d60148f2d00c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:56.186176 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac144c-4974-47cb-9e4c-d60148f2d00c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:56.197188 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d352f45-603d-4ac6-a840-683c1571c58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:56.199409 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d352f45-603d-4ac6-a840-683c1571c58a admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:56.200006 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d352f45-603d-4ac6-a840-683c1571c58a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:56.200514 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d352f45-603d-4ac6-a840-683c1571c58a admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:56.201857 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d352f45-603d-4ac6-a840-683c1571c58a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:56.207039 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d352f45-603d-4ac6-a840-683c1571c58a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:56.230106 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:56.279908 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1043/2085] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:26:55 2022] POST /identity/v3/projects => generated 379 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:26:56.284871 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b34464-46af-4352-9593-598afd9104bf None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:56.308130 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b34464-46af-4352-9593-598afd9104bf None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:56.456078 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59193bc5-5349-49c0-97ce-eebf6dd856b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:56.457682 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59193bc5-5349-49c0-97ce-eebf6dd856b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:56.458371 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59193bc5-5349-49c0-97ce-eebf6dd856b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:56.458858 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59193bc5-5349-49c0-97ce-eebf6dd856b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:56.461340 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59193bc5-5349-49c0-97ce-eebf6dd856b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:56.464941 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59193bc5-5349-49c0-97ce-eebf6dd856b9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:56.485029 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:56.515171 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1043/2086] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:26:56 2022] POST /identity/v3/users => generated 290 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:26:56.516838 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d352f45-603d-4ac6-a840-683c1571c58a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:56.550845 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d352f45-603d-4ac6-a840-683c1571c58a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:56.560414 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae6c02-9ec9-4548-8d8b-63e7128c5378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:56.561567 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae6c02-9ec9-4548-8d8b-63e7128c5378 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:56.562086 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae6c02-9ec9-4548-8d8b-63e7128c5378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:56.562592 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae6c02-9ec9-4548-8d8b-63e7128c5378 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:56.564110 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eae6c02-9ec9-4548-8d8b-63e7128c5378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:56.567319 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eae6c02-9ec9-4548-8d8b-63e7128c5378 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:56.593915 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1044/2087] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:26:56.600770 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59193bc5-5349-49c0-97ce-eebf6dd856b9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:56.621766 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59193bc5-5349-49c0-97ce-eebf6dd856b9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:56.628130 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859d187-0e6e-4534-968b-2157b5e951a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:56.629551 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859d187-0e6e-4534-968b-2157b5e951a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:56.630013 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859d187-0e6e-4534-968b-2157b5e951a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:56.630444 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859d187-0e6e-4534-968b-2157b5e951a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:56.632172 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f859d187-0e6e-4534-968b-2157b5e951a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:56.634552 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f859d187-0e6e-4534-968b-2157b5e951a0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:56.658424 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1044/2088] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:26:56.664307 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae6c02-9ec9-4548-8d8b-63e7128c5378 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:56.667027 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859d187-0e6e-4534-968b-2157b5e951a0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:56.667321 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2eae6c02-9ec9-4548-8d8b-63e7128c5378 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:56.692070 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859d187-0e6e-4534-968b-2157b5e951a0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:56.696485 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae6c02-9ec9-4548-8d8b-63e7128c5378 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:56.703347 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd34776-38f0-4191-8346-d762360bc5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:56.705127 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd34776-38f0-4191-8346-d762360bc5ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:56.705570 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd34776-38f0-4191-8346-d762360bc5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:56.706083 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd34776-38f0-4191-8346-d762360bc5ad admin admin] PATH_INFO: `/v3/projects/9a1617ab5e86475db7b124f52b488e5d/users/6aa176ba6788419c87001ac6589dad28/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:56.706249 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68436004-4f05-48b9-971f-f59414b23582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:56.707832 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68436004-4f05-48b9-971f-f59414b23582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:56.708355 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68436004-4f05-48b9-971f-f59414b23582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:56.708865 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68436004-4f05-48b9-971f-f59414b23582 admin admin] PATH_INFO: `/v3/users/defcbd87d502466684e1669e25962aed` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:56.714795 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68436004-4f05-48b9-971f-f59414b23582 admin admin] RBAC: Authorizing `identity:delete_user(user_id=defcbd87d502466684e1669e25962aed)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:56.719265 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68436004-4f05-48b9-971f-f59414b23582 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:56.747884 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd34776-38f0-4191-8346-d762360bc5ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a1617ab5e86475db7b124f52b488e5d, user_id=6aa176ba6788419c87001ac6589dad28, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:56.755400 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd34776-38f0-4191-8346-d762360bc5ad admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:56.790631 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1045/2089] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:26:56 2022] PUT /identity/v3/projects/9a1617ab5e86475db7b124f52b488e5d/users/6aa176ba6788419c87001ac6589dad28/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:26:56.806811 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c24ab-0f22-403c-a5a4-7765d055c70f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:56.807410 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c24ab-0f22-403c-a5a4-7765d055c70f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:56.808168 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c24ab-0f22-403c-a5a4-7765d055c70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:56.830435 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-68436004-4f05-48b9-971f-f59414b23582 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'defcbd87d502466684e1669e25962aed'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:56.849339 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-68436004-4f05-48b9-971f-f59414b23582 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'defcbd87d502466684e1669e25962aed'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:56.864776 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-68436004-4f05-48b9-971f-f59414b23582 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'defcbd87d502466684e1669e25962aed'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:56.866580 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-68436004-4f05-48b9-971f-f59414b23582 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'defcbd87d502466684e1669e25962aed'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:56.867906 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-68436004-4f05-48b9-971f-f59414b23582 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'defcbd87d502466684e1669e25962aed'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:56.875186 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-9f6c24ab-0f22-403c-a5a4-7765d055c70f None None] MFA Rules not processed for user `6aa176ba6788419c87001ac6589dad28`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:56.878809 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9f6c24ab-0f22-403c-a5a4-7765d055c70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:56.908185 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1045/2090] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:26:56 2022] DELETE /identity/v3/users/defcbd87d502466684e1669e25962aed => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:26:56.923197 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500f3f1-bcef-4f9f-98dd-a35ef2e56362 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:56.923197 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500f3f1-bcef-4f9f-98dd-a35ef2e56362 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:56.923197 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500f3f1-bcef-4f9f-98dd-a35ef2e56362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:56.962644 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b500f3f1-bcef-4f9f-98dd-a35ef2e56362 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:56.965903 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b500f3f1-bcef-4f9f-98dd-a35ef2e56362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:57.031688 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1046/2091] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:26:57.034108 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c24ab-0f22-403c-a5a4-7765d055c70f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:57.035966 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9f6c24ab-0f22-403c-a5a4-7765d055c70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:57.109669 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1046/2092] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:26:57.127505 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b500f3f1-bcef-4f9f-98dd-a35ef2e56362 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:57.130808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b500f3f1-bcef-4f9f-98dd-a35ef2e56362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:57.193846 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c24ab-0f22-403c-a5a4-7765d055c70f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:57.205490 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b9abaa-cdb6-4f0b-997a-434b7f6f182e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:57.207852 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9abaa-cdb6-4f0b-997a-434b7f6f182e admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:57.208490 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9abaa-cdb6-4f0b-997a-434b7f6f182e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:57.210051 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9abaa-cdb6-4f0b-997a-434b7f6f182e admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:57.212065 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b9abaa-cdb6-4f0b-997a-434b7f6f182e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:57.217902 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b9abaa-cdb6-4f0b-997a-434b7f6f182e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:57.271342 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b500f3f1-bcef-4f9f-98dd-a35ef2e56362 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:57.279794 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b1eb5-a6c9-40bb-8b2f-8b72fbb16f93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:57.281012 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b1eb5-a6c9-40bb-8b2f-8b72fbb16f93 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:57.281500 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b1eb5-a6c9-40bb-8b2f-8b72fbb16f93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:57.282039 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b1eb5-a6c9-40bb-8b2f-8b72fbb16f93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:57.294104 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1047/2093] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:26:57 2022] POST /identity/v3/projects => generated 377 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:26:57.295230 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:57.300682 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b9abaa-cdb6-4f0b-997a-434b7f6f182e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:57.307320 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b1eb5-a6c9-40bb-8b2f-8b72fbb16f93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:57.312057 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b1eb5-a6c9-40bb-8b2f-8b72fbb16f93 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:57.335927 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1047/2094] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:26:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:26:57.429796 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:57.512725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b9abaa-cdb6-4f0b-997a-434b7f6f182e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:57.530988 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c5171-726a-4b45-a756-3fa99dacaf92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:57.533348 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c5171-726a-4b45-a756-3fa99dacaf92 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:57.534062 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c5171-726a-4b45-a756-3fa99dacaf92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:57.534735 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c5171-726a-4b45-a756-3fa99dacaf92 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:57.535888 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c5171-726a-4b45-a756-3fa99dacaf92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:57.539618 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c5171-726a-4b45-a756-3fa99dacaf92 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:57.624104 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b1eb5-a6c9-40bb-8b2f-8b72fbb16f93 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:57.634710 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1048/2095] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:26:57 2022] POST /identity/v3/users => generated 295 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:26:57.646518 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b1eb5-a6c9-40bb-8b2f-8b72fbb16f93 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:57.655548 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b4675-3a89-41f2-be01-d9a440d3d093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:57.656970 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b4675-3a89-41f2-be01-d9a440d3d093 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:57.657466 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b4675-3a89-41f2-be01-d9a440d3d093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:57.658007 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b4675-3a89-41f2-be01-d9a440d3d093 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:57.659042 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b4675-3a89-41f2-be01-d9a440d3d093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:57.662034 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b4675-3a89-41f2-be01-d9a440d3d093 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:57.698642 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c5171-726a-4b45-a756-3fa99dacaf92 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:57.701437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1048/2096] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:26:57.722279 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c5171-726a-4b45-a756-3fa99dacaf92 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:57.731442 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53cbaa-f115-4a83-b736-5eaaa3920dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:57.734663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53cbaa-f115-4a83-b736-5eaaa3920dcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:57.735469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53cbaa-f115-4a83-b736-5eaaa3920dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:57.736067 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53cbaa-f115-4a83-b736-5eaaa3920dcf admin admin] PATH_INFO: `/v3/projects/181944174a4d4d1cb511f28c774de320/users/69b4c6c09c4741939b13309ccbe9783f/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:57.780292 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c53cbaa-f115-4a83-b736-5eaaa3920dcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=181944174a4d4d1cb511f28c774de320, user_id=69b4c6c09c4741939b13309ccbe9783f, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:57.797968 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c53cbaa-f115-4a83-b736-5eaaa3920dcf admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:57.857098 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1049/2097] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:26:57 2022] PUT /identity/v3/projects/181944174a4d4d1cb511f28c774de320/users/69b4c6c09c4741939b13309ccbe9783f/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:26:57.862181 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b4675-3a89-41f2-be01-d9a440d3d093 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:57.886414 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b4675-3a89-41f2-be01-d9a440d3d093 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:58.041658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e51e45f-9a2e-4bd0-88a4-703ff4f0f0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:58.043284 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51e45f-9a2e-4bd0-88a4-703ff4f0f0d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:58.043939 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51e45f-9a2e-4bd0-88a4-703ff4f0f0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:58.044534 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51e45f-9a2e-4bd0-88a4-703ff4f0f0d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:58.046394 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e51e45f-9a2e-4bd0-88a4-703ff4f0f0d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:58.050396 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e51e45f-9a2e-4bd0-88a4-703ff4f0f0d2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:58.090412 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1049/2098] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:26:58.092543 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53cbaa-f115-4a83-b736-5eaaa3920dcf admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:58.115549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53cbaa-f115-4a83-b736-5eaaa3920dcf admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:58.138746 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe95d0d-35ea-4cab-b610-f7b987f0b56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:58.143621 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe95d0d-35ea-4cab-b610-f7b987f0b56d admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:58.143621 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe95d0d-35ea-4cab-b610-f7b987f0b56d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:58.143621 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe95d0d-35ea-4cab-b610-f7b987f0b56d admin admin] PATH_INFO: `/v3/projects/181944174a4d4d1cb511f28c774de320/users/69b4c6c09c4741939b13309ccbe9783f/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:58.146888 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe95d0d-35ea-4cab-b610-f7b987f0b56d admin admin] RBAC: Authorizing `identity:create_grant(project_id=181944174a4d4d1cb511f28c774de320, user_id=69b4c6c09c4741939b13309ccbe9783f, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:58.153657 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe95d0d-35ea-4cab-b610-f7b987f0b56d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:58.194886 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1050/2099] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:26:58 2022] PUT /identity/v3/projects/181944174a4d4d1cb511f28c774de320/users/69b4c6c09c4741939b13309ccbe9783f/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:26:58.218360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e51e45f-9a2e-4bd0-88a4-703ff4f0f0d2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:58.222381 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8e51e45f-9a2e-4bd0-88a4-703ff4f0f0d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:58.227860 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f312ab-2819-4317-83d3-a10ef2fbf867 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:58.228798 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f312ab-2819-4317-83d3-a10ef2fbf867 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:58.229658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f312ab-2819-4317-83d3-a10ef2fbf867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:58.255131 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e51e45f-9a2e-4bd0-88a4-703ff4f0f0d2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:58.316942 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-64f312ab-2819-4317-83d3-a10ef2fbf867 None None] MFA Rules not processed for user `69b4c6c09c4741939b13309ccbe9783f`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:58.320406 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-64f312ab-2819-4317-83d3-a10ef2fbf867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:58.437054 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7cb539-3903-423a-be7b-01928f06564a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:58.438324 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7cb539-3903-423a-be7b-01928f06564a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:58.439129 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7cb539-3903-423a-be7b-01928f06564a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:58.439434 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7cb539-3903-423a-be7b-01928f06564a admin admin] PATH_INFO: `/v3/projects/1ee9720cbdf04d17b7c0b08fe373d203` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:58.441378 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7cb539-3903-423a-be7b-01928f06564a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ee9720cbdf04d17b7c0b08fe373d203)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:58.445452 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7cb539-3903-423a-be7b-01928f06564a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:58.505034 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1051/2100] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:58 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:26:58.526871 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f312ab-2819-4317-83d3-a10ef2fbf867 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:58.529563 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-64f312ab-2819-4317-83d3-a10ef2fbf867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:58.605618 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9b7cb539-3903-423a-be7b-01928f06564a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ee9720cbdf04d17b7c0b08fe373d203', 'request_id': 'req-9b7cb539-3903-423a-be7b-01928f06564a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:58.631131 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9b7cb539-3903-423a-be7b-01928f06564a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ee9720cbdf04d17b7c0b08fe373d203', 'request_id': 'req-9b7cb539-3903-423a-be7b-01928f06564a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:58.650026 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9b7cb539-3903-423a-be7b-01928f06564a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ee9720cbdf04d17b7c0b08fe373d203', 'request_id': 'req-9b7cb539-3903-423a-be7b-01928f06564a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:58.684954 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9b7cb539-3903-423a-be7b-01928f06564a admin admin] The token cache is being invalidate because project 1ee9720cbdf04d17b7c0b08fe373d203 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:26:58.685550 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9b7cb539-3903-423a-be7b-01928f06564a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:58.688069 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9b7cb539-3903-423a-be7b-01928f06564a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:26:58.690907 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1050/2101] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:26:58 2022] DELETE /identity/v3/projects/1ee9720cbdf04d17b7c0b08fe373d203 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:26:58.699117 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d6e15-f255-4a26-9a02-f90431ed290c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:58.699876 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d6e15-f255-4a26-9a02-f90431ed290c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:58.700365 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d6e15-f255-4a26-9a02-f90431ed290c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:58.707898 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f312ab-2819-4317-83d3-a10ef2fbf867 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:58.751862 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6b6d6e15-f255-4a26-9a02-f90431ed290c None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:58.755097 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6b6d6e15-f255-4a26-9a02-f90431ed290c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:58.765086 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:58.784195 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:58.842849 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23122b66-ac0d-4015-a477-cbd16b6189bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:58.844138 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23122b66-ac0d-4015-a477-cbd16b6189bc service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:58.844661 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23122b66-ac0d-4015-a477-cbd16b6189bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:58.845182 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23122b66-ac0d-4015-a477-cbd16b6189bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:58.849021 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-23122b66-ac0d-4015-a477-cbd16b6189bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:58.929381 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1051/2102] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:26:58.945098 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c3668-d84b-4178-9743-2115ff1f956b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:58.945672 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c3668-d84b-4178-9743-2115ff1f956b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:58.946217 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c3668-d84b-4178-9743-2115ff1f956b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:58.995378 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b86c3668-d84b-4178-9743-2115ff1f956b None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:26:58.997008 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b86c3668-d84b-4178-9743-2115ff1f956b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:59.010402 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1052/2103] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:26:59.017755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c3668-d84b-4178-9743-2115ff1f956b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:59.040028 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c3668-d84b-4178-9743-2115ff1f956b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:59.051676 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b46a8-13e7-4ec7-be27-4177be94efdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:59.053515 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b46a8-13e7-4ec7-be27-4177be94efdd admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:59.055451 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b46a8-13e7-4ec7-be27-4177be94efdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:59.056106 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b46a8-13e7-4ec7-be27-4177be94efdd admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:59.058193 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b46a8-13e7-4ec7-be27-4177be94efdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:26:59.058726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b46a8-13e7-4ec7-be27-4177be94efdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:59.060967 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b46a8-13e7-4ec7-be27-4177be94efdd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:59.081936 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1053/2104] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:26:59.085169 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23122b66-ac0d-4015-a477-cbd16b6189bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:59.090764 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23122b66-ac0d-4015-a477-cbd16b6189bc service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:59.092210 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b46a8-13e7-4ec7-be27-4177be94efdd admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:59.108449 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b46a8-13e7-4ec7-be27-4177be94efdd admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:59.117219 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54741c9-b343-45db-be2c-7186346b33bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:59.118507 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54741c9-b343-45db-be2c-7186346b33bf admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:59.119141 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54741c9-b343-45db-be2c-7186346b33bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:59.119803 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54741c9-b343-45db-be2c-7186346b33bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:59.121261 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54741c9-b343-45db-be2c-7186346b33bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:59.123408 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1052/2105] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:26:58 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:26:59.130842 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54741c9-b343-45db-be2c-7186346b33bf admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:59.220562 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23122b66-ac0d-4015-a477-cbd16b6189bc service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:59.222613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-23122b66-ac0d-4015-a477-cbd16b6189bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:59.234742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1054/2106] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:26:59 2022] POST /identity/v3/projects => generated 373 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:26:59.236593 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54741c9-b343-45db-be2c-7186346b33bf admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:59.248043 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23122b66-ac0d-4015-a477-cbd16b6189bc service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:59.393223 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b80434-848c-4ec8-90fe-8a78c2010b1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:59.395351 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b80434-848c-4ec8-90fe-8a78c2010b1e service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:59.397984 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b80434-848c-4ec8-90fe-8a78c2010b1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:59.400005 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b80434-848c-4ec8-90fe-8a78c2010b1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:59.404662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e2b80434-848c-4ec8-90fe-8a78c2010b1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:26:59.481637 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54741c9-b343-45db-be2c-7186346b33bf admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:59.494363 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9c68a-237b-49c9-8167-8e35f197bccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:59.497752 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9c68a-237b-49c9-8167-8e35f197bccd admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:59.498871 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9c68a-237b-49c9-8167-8e35f197bccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:59.499729 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9c68a-237b-49c9-8167-8e35f197bccd admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:59.501412 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb9c68a-237b-49c9-8167-8e35f197bccd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:59.508017 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb9c68a-237b-49c9-8167-8e35f197bccd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:59.547272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b80434-848c-4ec8-90fe-8a78c2010b1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:59.550951 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b80434-848c-4ec8-90fe-8a78c2010b1e service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:59.571158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1053/2107] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:26:59 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:26:59.575824 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1055/2108] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:26:59 2022] POST /identity/v3/users => generated 287 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:26:59.578579 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b80434-848c-4ec8-90fe-8a78c2010b1e service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:59.603052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b80434-848c-4ec8-90fe-8a78c2010b1e service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:59.612689 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186af82d-f1e5-487a-925d-8ce7033f56d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:59.614470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186af82d-f1e5-487a-925d-8ce7033f56d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:59.614470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186af82d-f1e5-487a-925d-8ce7033f56d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:59.615830 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186af82d-f1e5-487a-925d-8ce7033f56d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:59.616603 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186af82d-f1e5-487a-925d-8ce7033f56d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:59.619780 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186af82d-f1e5-487a-925d-8ce7033f56d1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:59.660875 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1054/2109] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:26:59.663904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9c68a-237b-49c9-8167-8e35f197bccd admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:59.688315 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9c68a-237b-49c9-8167-8e35f197bccd admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:59.699436 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c34ddd-e406-462a-8cd7-918679ed88f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:59.700699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c34ddd-e406-462a-8cd7-918679ed88f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:59.701430 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c34ddd-e406-462a-8cd7-918679ed88f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:59.701632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c34ddd-e406-462a-8cd7-918679ed88f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:59.702988 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c34ddd-e406-462a-8cd7-918679ed88f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:59.707176 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c34ddd-e406-462a-8cd7-918679ed88f6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:59.734463 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1056/2110] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:26:59.741478 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186af82d-f1e5-487a-925d-8ce7033f56d1 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:26:59.766064 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186af82d-f1e5-487a-925d-8ce7033f56d1 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:26:59.777470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18eaa5-1ab2-4367-bc07-6f3990c961d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:26:59.779376 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18eaa5-1ab2-4367-bc07-6f3990c961d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:59.780247 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18eaa5-1ab2-4367-bc07-6f3990c961d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:59.780855 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18eaa5-1ab2-4367-bc07-6f3990c961d6 admin admin] PATH_INFO: `/v3/projects/e46105ba7eda4d4fab37f228b3ee7301/users/4912e51f29cb4cb59c542751829dd9b6/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:26:59.830236 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a18eaa5-1ab2-4367-bc07-6f3990c961d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e46105ba7eda4d4fab37f228b3ee7301, user_id=4912e51f29cb4cb59c542751829dd9b6, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:26:59.836180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a18eaa5-1ab2-4367-bc07-6f3990c961d6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:26:59.864278 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:26:59.899964 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1055/2111] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:26:59 2022] PUT /identity/v3/projects/e46105ba7eda4d4fab37f228b3ee7301/users/4912e51f29cb4cb59c542751829dd9b6/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:26:59.913218 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9b582-b750-41af-8813-04d59694420b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:26:59.913619 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9b582-b750-41af-8813-04d59694420b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:26:59.914286 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9b582-b750-41af-8813-04d59694420b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:00.007471 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-fee9b582-b750-41af-8813-04d59694420b None None] MFA Rules not processed for user `4912e51f29cb4cb59c542751829dd9b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:00.013615 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fee9b582-b750-41af-8813-04d59694420b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:00.106980 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:27:00.215889 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1057/2112] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:26:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:27:00.234301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18eaa5-1ab2-4367-bc07-6f3990c961d6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:00.237626 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3a18eaa5-1ab2-4367-bc07-6f3990c961d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:00.419091 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18eaa5-1ab2-4367-bc07-6f3990c961d6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:00.439224 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab07c74-4712-4e6b-9b3b-78472705a952 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:00.440740 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab07c74-4712-4e6b-9b3b-78472705a952 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:00.441539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab07c74-4712-4e6b-9b3b-78472705a952 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:00.442120 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab07c74-4712-4e6b-9b3b-78472705a952 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:00.520315 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab07c74-4712-4e6b-9b3b-78472705a952 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:00.526470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab07c74-4712-4e6b-9b3b-78472705a952 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:00.564672 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1056/2113] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:27:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:27:00.655928 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee9b582-b750-41af-8813-04d59694420b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:00.858394 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee9b582-b750-41af-8813-04d59694420b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:00.903356 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc65990-02b2-4a83-956f-abae33d7da3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:00.904205 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc65990-02b2-4a83-956f-abae33d7da3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:00.904735 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc65990-02b2-4a83-956f-abae33d7da3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:00.905223 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc65990-02b2-4a83-956f-abae33d7da3d admin admin] PATH_INFO: `/v3/users/4912e51f29cb4cb59c542751829dd9b6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:00.907926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc65990-02b2-4a83-956f-abae33d7da3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4912e51f29cb4cb59c542751829dd9b6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:00.911990 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc65990-02b2-4a83-956f-abae33d7da3d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:01.087436 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ccc65990-02b2-4a83-956f-abae33d7da3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4912e51f29cb4cb59c542751829dd9b6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:01.108913 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ccc65990-02b2-4a83-956f-abae33d7da3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4912e51f29cb4cb59c542751829dd9b6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:01.125391 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ccc65990-02b2-4a83-956f-abae33d7da3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4912e51f29cb4cb59c542751829dd9b6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:01.126272 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ccc65990-02b2-4a83-956f-abae33d7da3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4912e51f29cb4cb59c542751829dd9b6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:01.127596 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ccc65990-02b2-4a83-956f-abae33d7da3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4912e51f29cb4cb59c542751829dd9b6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:01.191024 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1058/2114] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:27:00 2022] DELETE /identity/v3/users/4912e51f29cb4cb59c542751829dd9b6 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:27:01.199307 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5254244-c568-486d-ab23-b1c219b186d9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:01.200931 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5254244-c568-486d-ab23-b1c219b186d9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:01.201614 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5254244-c568-486d-ab23-b1c219b186d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:01.242501 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e5254244-c568-486d-ab23-b1c219b186d9 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:01.244811 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e5254244-c568-486d-ab23-b1c219b186d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:01.426449 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1057/2115] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:27:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:27:01.435549 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc65990-02b2-4a83-956f-abae33d7da3d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:01.437850 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ccc65990-02b2-4a83-956f-abae33d7da3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:01.675996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc65990-02b2-4a83-956f-abae33d7da3d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:01.678919 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb6a6b-5b86-4050-89dd-d32d2b3f0b87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:01.680527 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb6a6b-5b86-4050-89dd-d32d2b3f0b87 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:01.681260 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb6a6b-5b86-4050-89dd-d32d2b3f0b87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:01.682041 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb6a6b-5b86-4050-89dd-d32d2b3f0b87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:01.705036 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb6a6b-5b86-4050-89dd-d32d2b3f0b87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:01.708628 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb6a6b-5b86-4050-89dd-d32d2b3f0b87 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:01.725837 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:27:01.753216 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1059/2116] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:27:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:27:02.861512 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5254244-c568-486d-ab23-b1c219b186d9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:02.863732 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e5254244-c568-486d-ab23-b1c219b186d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:02.884573 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5254244-c568-486d-ab23-b1c219b186d9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:02.891818 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eaab43-3614-459e-835c-470de588875e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:02.892735 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eaab43-3614-459e-835c-470de588875e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:02.893022 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eaab43-3614-459e-835c-470de588875e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:02.893302 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eaab43-3614-459e-835c-470de588875e admin admin] PATH_INFO: `/v3/projects/e46105ba7eda4d4fab37f228b3ee7301` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:02.894745 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eaab43-3614-459e-835c-470de588875e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e46105ba7eda4d4fab37f228b3ee7301)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:02.897359 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eaab43-3614-459e-835c-470de588875e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:02.988498 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-75eaab43-3614-459e-835c-470de588875e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e46105ba7eda4d4fab37f228b3ee7301', 'request_id': 'req-75eaab43-3614-459e-835c-470de588875e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:03.005109 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-75eaab43-3614-459e-835c-470de588875e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e46105ba7eda4d4fab37f228b3ee7301', 'request_id': 'req-75eaab43-3614-459e-835c-470de588875e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:03.019848 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-75eaab43-3614-459e-835c-470de588875e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e46105ba7eda4d4fab37f228b3ee7301', 'request_id': 'req-75eaab43-3614-459e-835c-470de588875e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:03.048603 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-75eaab43-3614-459e-835c-470de588875e admin admin] The token cache is being invalidate because project e46105ba7eda4d4fab37f228b3ee7301 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:27:03.049470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-75eaab43-3614-459e-835c-470de588875e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:03.050874 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-75eaab43-3614-459e-835c-470de588875e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:03.054815 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1058/2117] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:27:02 2022] DELETE /identity/v3/projects/e46105ba7eda4d4fab37f228b3ee7301 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:27:03.064915 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f1c08-5ec9-4f93-a380-c25cdd951c51 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:03.065562 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f1c08-5ec9-4f93-a380-c25cdd951c51 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:03.066153 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f1c08-5ec9-4f93-a380-c25cdd951c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:03.108612 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-aa7f1c08-5ec9-4f93-a380-c25cdd951c51 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:03.110608 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-aa7f1c08-5ec9-4f93-a380-c25cdd951c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:03.233432 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1060/2118] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:27:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:27:03.242708 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47d9fd-74e2-41fd-a80c-b60767e69885 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:03.243450 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47d9fd-74e2-41fd-a80c-b60767e69885 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:03.243941 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47d9fd-74e2-41fd-a80c-b60767e69885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:03.276890 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-df47d9fd-74e2-41fd-a80c-b60767e69885 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:03.278477 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-df47d9fd-74e2-41fd-a80c-b60767e69885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:03.288452 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1059/2119] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:27:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:27:03.294022 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f1c08-5ec9-4f93-a380-c25cdd951c51 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:03.315052 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f1c08-5ec9-4f93-a380-c25cdd951c51 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:03.322185 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddfa3bb-4e03-41a8-b1ab-d53a9233974b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:03.323317 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddfa3bb-4e03-41a8-b1ab-d53a9233974b admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:03.324020 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddfa3bb-4e03-41a8-b1ab-d53a9233974b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:03.324215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddfa3bb-4e03-41a8-b1ab-d53a9233974b admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:03.325474 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddfa3bb-4e03-41a8-b1ab-d53a9233974b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:27:03.326110 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddfa3bb-4e03-41a8-b1ab-d53a9233974b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:03.329052 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddfa3bb-4e03-41a8-b1ab-d53a9233974b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:03.349440 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995ac37-7f12-46d7-9d94-ea413bd0c6b5 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:03.350177 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995ac37-7f12-46d7-9d94-ea413bd0c6b5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:03.351610 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995ac37-7f12-46d7-9d94-ea413bd0c6b5 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:03.351799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1061/2120] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:27:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:27:03.353495 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddfa3bb-4e03-41a8-b1ab-d53a9233974b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:03.354483 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1060/2121] 10.222.0.18 () {54 vars in 921 bytes} [Mon Jul 25 19:27:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:27:03.370908 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddfa3bb-4e03-41a8-b1ab-d53a9233974b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:03.377239 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d6e6d-8819-404e-96e0-9df84bca0be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:03.377546 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d6e6d-8819-404e-96e0-9df84bca0be7 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:03.377769 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d6e6d-8819-404e-96e0-9df84bca0be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:03.378126 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d6e6d-8819-404e-96e0-9df84bca0be7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:03.378917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011d6e6d-8819-404e-96e0-9df84bca0be7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:03.381531 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011d6e6d-8819-404e-96e0-9df84bca0be7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:03.387361 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e1546-e614-45fa-be51-eb279aae0760 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:03.387361 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e1546-e614-45fa-be51-eb279aae0760 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:03.387361 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e1546-e614-45fa-be51-eb279aae0760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:03.441864 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-854e1546-e614-45fa-be51-eb279aae0760 None None] MFA Rules not processed for user `0422d6695bf14925bf0f993cb16e64ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:03.444738 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-854e1546-e614-45fa-be51-eb279aae0760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:03.455792 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1062/2122] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:27:03 2022] POST /identity/v3/projects => generated 375 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:27:03.456961 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d6e6d-8819-404e-96e0-9df84bca0be7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:03.459664 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-011d6e6d-8819-404e-96e0-9df84bca0be7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:03.480475 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:27:03.606310 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1061/2123] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Jul 25 19:27:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:27:03.607613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e1546-e614-45fa-be51-eb279aae0760 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:03.623353 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d6e6d-8819-404e-96e0-9df84bca0be7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:03.629745 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e1546-e614-45fa-be51-eb279aae0760 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:03.636083 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594cf6c-ffb4-48ec-a6db-ea34187355a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:03.637049 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594cf6c-ffb4-48ec-a6db-ea34187355a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:03.637459 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594cf6c-ffb4-48ec-a6db-ea34187355a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:03.637843 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594cf6c-ffb4-48ec-a6db-ea34187355a1 admin admin] PATH_INFO: `/v3/users/30efc9bc66ce498d9d94980d85152694` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:03.639050 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74647a1-2553-4fb9-8dc1-93df48b16969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:03.639358 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f594cf6c-ffb4-48ec-a6db-ea34187355a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30efc9bc66ce498d9d94980d85152694)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:03.639927 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74647a1-2553-4fb9-8dc1-93df48b16969 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:03.640286 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74647a1-2553-4fb9-8dc1-93df48b16969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:03.640640 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74647a1-2553-4fb9-8dc1-93df48b16969 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:03.641426 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74647a1-2553-4fb9-8dc1-93df48b16969 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:03.641595 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f594cf6c-ffb4-48ec-a6db-ea34187355a1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:03.643652 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74647a1-2553-4fb9-8dc1-93df48b16969 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:03.698190 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1062/2124] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:27:03 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:27:03.699350 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74647a1-2553-4fb9-8dc1-93df48b16969 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:03.706276 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f74647a1-2553-4fb9-8dc1-93df48b16969 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:03.729276 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74647a1-2553-4fb9-8dc1-93df48b16969 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:03.738015 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec0e43-4d4f-431f-bb19-e4dddf596296 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0422d6695bf14925bf0f993cb16e64ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:03.739569 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec0e43-4d4f-431f-bb19-e4dddf596296 service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:03.739947 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec0e43-4d4f-431f-bb19-e4dddf596296 service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:03.740934 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec0e43-4d4f-431f-bb19-e4dddf596296 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:03.762295 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ec0e43-4d4f-431f-bb19-e4dddf596296 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:03.765045 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ec0e43-4d4f-431f-bb19-e4dddf596296 service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:03.767416 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f594cf6c-ffb4-48ec-a6db-ea34187355a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30efc9bc66ce498d9d94980d85152694'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:03.783856 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1063/2125] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:27:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:27:03.785036 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec0e43-4d4f-431f-bb19-e4dddf596296 service placement] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:03.804576 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec0e43-4d4f-431f-bb19-e4dddf596296 service placement] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:03.816276 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f594cf6c-ffb4-48ec-a6db-ea34187355a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30efc9bc66ce498d9d94980d85152694'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:03.836810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8d9a1-bafb-40a3-8f0c-831b5c43d5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:03.838592 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f594cf6c-ffb4-48ec-a6db-ea34187355a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30efc9bc66ce498d9d94980d85152694'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:03.839947 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f594cf6c-ffb4-48ec-a6db-ea34187355a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30efc9bc66ce498d9d94980d85152694'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:03.840698 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f594cf6c-ffb4-48ec-a6db-ea34187355a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30efc9bc66ce498d9d94980d85152694'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:03.844064 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:27:03.852997 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8d9a1-bafb-40a3-8f0c-831b5c43d5e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:03.855551 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8d9a1-bafb-40a3-8f0c-831b5c43d5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:03.856098 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8d9a1-bafb-40a3-8f0c-831b5c43d5e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:03.861066 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c8d9a1-bafb-40a3-8f0c-831b5c43d5e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:03.865856 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c8d9a1-bafb-40a3-8f0c-831b5c43d5e7 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:03.876855 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1063/2126] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:27:03 2022] DELETE /identity/v3/users/30efc9bc66ce498d9d94980d85152694 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:27:03.905753 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1064/2127] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:27:03.910430 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594cf6c-ffb4-48ec-a6db-ea34187355a1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:03.912035 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f594cf6c-ffb4-48ec-a6db-ea34187355a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:03.966130 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8d9a1-bafb-40a3-8f0c-831b5c43d5e7 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:03.969089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a1c8d9a1-bafb-40a3-8f0c-831b5c43d5e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:04.020010 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8d9a1-bafb-40a3-8f0c-831b5c43d5e7 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:04.082737 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594cf6c-ffb4-48ec-a6db-ea34187355a1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:04.090213 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f78eb-56b7-4d7b-8518-c012321f66d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:04.090981 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f78eb-56b7-4d7b-8518-c012321f66d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:04.091271 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f78eb-56b7-4d7b-8518-c012321f66d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:04.091599 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f78eb-56b7-4d7b-8518-c012321f66d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:04.092374 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f78eb-56b7-4d7b-8518-c012321f66d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:04.094310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f78eb-56b7-4d7b-8518-c012321f66d6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:04.108880 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1064/2128] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:27:04.116167 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f78eb-56b7-4d7b-8518-c012321f66d6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:04.135067 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f78eb-56b7-4d7b-8518-c012321f66d6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:04.144193 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9351216f-3ca9-467a-86e6-75cfe6bd3e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:04.145658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9351216f-3ca9-467a-86e6-75cfe6bd3e22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:04.146108 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9351216f-3ca9-467a-86e6-75cfe6bd3e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:04.146398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9351216f-3ca9-467a-86e6-75cfe6bd3e22 admin admin] PATH_INFO: `/v3/projects/a092c7d1ac694eab812fe205ccbe5434/users/dc63fdeacd514eac8e4a5882190f8d66/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:04.179240 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a437a-ae4d-4598-99b6-f9c456e4eccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:04.180396 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a437a-ae4d-4598-99b6-f9c456e4eccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:04.180855 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a437a-ae4d-4598-99b6-f9c456e4eccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:04.181754 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a437a-ae4d-4598-99b6-f9c456e4eccc admin admin] PATH_INFO: `/v3/users/f9197a67739a40a4be3c9b8a65b2b1d6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:04.186237 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151a437a-ae4d-4598-99b6-f9c456e4eccc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9197a67739a40a4be3c9b8a65b2b1d6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:04.190443 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151a437a-ae4d-4598-99b6-f9c456e4eccc admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:04.199227 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9351216f-3ca9-467a-86e6-75cfe6bd3e22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a092c7d1ac694eab812fe205ccbe5434, user_id=dc63fdeacd514eac8e4a5882190f8d66, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:04.202688 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9351216f-3ca9-467a-86e6-75cfe6bd3e22 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:04.234947 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1065/2129] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:27:04 2022] PUT /identity/v3/projects/a092c7d1ac694eab812fe205ccbe5434/users/dc63fdeacd514eac8e4a5882190f8d66/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:27:04.246936 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbd08e-8694-4853-9294-d43254b26344 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:04.247244 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbd08e-8694-4853-9294-d43254b26344 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:04.247489 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbd08e-8694-4853-9294-d43254b26344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:04.324234 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c6dbd08e-8694-4853-9294-d43254b26344 None None] MFA Rules not processed for user `dc63fdeacd514eac8e4a5882190f8d66`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:04.325737 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c6dbd08e-8694-4853-9294-d43254b26344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:04.360907 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-151a437a-ae4d-4598-99b6-f9c456e4eccc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9197a67739a40a4be3c9b8a65b2b1d6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:04.372450 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-151a437a-ae4d-4598-99b6-f9c456e4eccc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9197a67739a40a4be3c9b8a65b2b1d6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:04.384547 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-151a437a-ae4d-4598-99b6-f9c456e4eccc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9197a67739a40a4be3c9b8a65b2b1d6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:04.385403 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-151a437a-ae4d-4598-99b6-f9c456e4eccc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9197a67739a40a4be3c9b8a65b2b1d6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:04.403658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:27:04.413683 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-151a437a-ae4d-4598-99b6-f9c456e4eccc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9197a67739a40a4be3c9b8a65b2b1d6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:04.431782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1065/2130] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:27:03 2022] DELETE /identity/v3/users/f9197a67739a40a4be3c9b8a65b2b1d6 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:27:04.433627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a437a-ae4d-4598-99b6-f9c456e4eccc admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:04.435239 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-151a437a-ae4d-4598-99b6-f9c456e4eccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:04.439035 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1066/2131] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:27:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:27:04.441541 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391dc96-7c23-4182-aefe-05506f990e64 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:04.442007 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391dc96-7c23-4182-aefe-05506f990e64 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:04.442468 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391dc96-7c23-4182-aefe-05506f990e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:04.475686 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f391dc96-7c23-4182-aefe-05506f990e64 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:04.478001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f391dc96-7c23-4182-aefe-05506f990e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:04.551710 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a437a-ae4d-4598-99b6-f9c456e4eccc admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:04.558232 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:27:04.558590 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f587c2-214d-4af8-be6a-eba909ffd091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:04.560236 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f587c2-214d-4af8-be6a-eba909ffd091 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:04.560733 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f587c2-214d-4af8-be6a-eba909ffd091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:04.561399 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f587c2-214d-4af8-be6a-eba909ffd091 admin admin] PATH_INFO: `/v3/projects/a6744feae17b4b32a60e06d373feb52d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:04.563631 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f587c2-214d-4af8-be6a-eba909ffd091 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6744feae17b4b32a60e06d373feb52d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:04.566609 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f587c2-214d-4af8-be6a-eba909ffd091 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:04.595782 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1067/2132] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:27:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:27:04.596880 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391dc96-7c23-4182-aefe-05506f990e64 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:04.598429 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f391dc96-7c23-4182-aefe-05506f990e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:04.662902 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f3f587c2-214d-4af8-be6a-eba909ffd091 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6744feae17b4b32a60e06d373feb52d', 'request_id': 'req-f3f587c2-214d-4af8-be6a-eba909ffd091'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:04.676870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f3f587c2-214d-4af8-be6a-eba909ffd091 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6744feae17b4b32a60e06d373feb52d', 'request_id': 'req-f3f587c2-214d-4af8-be6a-eba909ffd091'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:04.688892 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f3f587c2-214d-4af8-be6a-eba909ffd091 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6744feae17b4b32a60e06d373feb52d', 'request_id': 'req-f3f587c2-214d-4af8-be6a-eba909ffd091'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:04.694428 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391dc96-7c23-4182-aefe-05506f990e64 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:04.711204 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f3f587c2-214d-4af8-be6a-eba909ffd091 admin admin] The token cache is being invalidate because project a6744feae17b4b32a60e06d373feb52d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:27:04.711650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f3f587c2-214d-4af8-be6a-eba909ffd091 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:04.713127 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f3f587c2-214d-4af8-be6a-eba909ffd091 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:04.717053 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1066/2133] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:27:04 2022] DELETE /identity/v3/projects/a6744feae17b4b32a60e06d373feb52d => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:27:04.718253 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f587c2-214d-4af8-be6a-eba909ffd091 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:04.722278 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f3f587c2-214d-4af8-be6a-eba909ffd091 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:04.790501 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34131d65-0ece-40fd-99b2-0f1634352674 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:04.791451 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34131d65-0ece-40fd-99b2-0f1634352674 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:04.791905 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34131d65-0ece-40fd-99b2-0f1634352674 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:04.792282 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34131d65-0ece-40fd-99b2-0f1634352674 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:04.799302 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-34131d65-0ece-40fd-99b2-0f1634352674 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:04.816460 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f587c2-214d-4af8-be6a-eba909ffd091 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:04.821229 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac524a85-d905-4141-9d09-1cbbb3b0cc16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:04.822081 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac524a85-d905-4141-9d09-1cbbb3b0cc16 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:04.822376 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac524a85-d905-4141-9d09-1cbbb3b0cc16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:04.822655 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac524a85-d905-4141-9d09-1cbbb3b0cc16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:04.829588 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ac524a85-d905-4141-9d09-1cbbb3b0cc16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:04.907656 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34131d65-0ece-40fd-99b2-0f1634352674 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:04.909489 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34131d65-0ece-40fd-99b2-0f1634352674 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:04.922184 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:27:04.927186 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1068/2134] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:27:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:27:04.930850 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2188f89-a9da-445a-bef3-9999db06219b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:04.931301 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2188f89-a9da-445a-bef3-9999db06219b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:04.931837 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2188f89-a9da-445a-bef3-9999db06219b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:04.968988 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d2188f89-a9da-445a-bef3-9999db06219b None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:04.971563 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d2188f89-a9da-445a-bef3-9999db06219b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:05.017138 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac524a85-d905-4141-9d09-1cbbb3b0cc16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:05.023226 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac524a85-d905-4141-9d09-1cbbb3b0cc16 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:05.055180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1067/2135] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:27:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:27:05.092945 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:27:05.110421 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1069/2136] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:27:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:27:05.125389 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb1c11-f103-48b8-b815-88caae4001d9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:05.125781 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb1c11-f103-48b8-b815-88caae4001d9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:05.126266 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb1c11-f103-48b8-b815-88caae4001d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:05.164051 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3bbb1c11-f103-48b8-b815-88caae4001d9 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:05.166504 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3bbb1c11-f103-48b8-b815-88caae4001d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:05.211342 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1068/2137] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:27:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:27:05.212347 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2188f89-a9da-445a-bef3-9999db06219b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:05.233136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2188f89-a9da-445a-bef3-9999db06219b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:05.245699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62792a97-cd5a-42d2-bbec-fff4f949ac78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:05.247080 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62792a97-cd5a-42d2-bbec-fff4f949ac78 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:05.247553 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62792a97-cd5a-42d2-bbec-fff4f949ac78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:05.247999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62792a97-cd5a-42d2-bbec-fff4f949ac78 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:05.249375 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62792a97-cd5a-42d2-bbec-fff4f949ac78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:27:05.249757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62792a97-cd5a-42d2-bbec-fff4f949ac78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:05.252207 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62792a97-cd5a-42d2-bbec-fff4f949ac78 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:05.276402 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1070/2138] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:27:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:27:05.286148 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb1c11-f103-48b8-b815-88caae4001d9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:05.312338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb1c11-f103-48b8-b815-88caae4001d9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:05.324593 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51108b7-b71c-49e8-aa50-0d2ab34d6677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:05.326067 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51108b7-b71c-49e8-aa50-0d2ab34d6677 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:05.326679 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51108b7-b71c-49e8-aa50-0d2ab34d6677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:05.327316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51108b7-b71c-49e8-aa50-0d2ab34d6677 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:05.329637 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51108b7-b71c-49e8-aa50-0d2ab34d6677 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:05.333952 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51108b7-b71c-49e8-aa50-0d2ab34d6677 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:05.434313 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1069/2139] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:27:05 2022] POST /identity/v3/projects => generated 377 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:27:05.440584 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62792a97-cd5a-42d2-bbec-fff4f949ac78 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:05.457309 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62792a97-cd5a-42d2-bbec-fff4f949ac78 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:05.606673 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dbe1a4-75bb-4b95-96ca-3c38038cfffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:05.608885 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbe1a4-75bb-4b95-96ca-3c38038cfffe admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:05.608885 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbe1a4-75bb-4b95-96ca-3c38038cfffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:05.609570 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbe1a4-75bb-4b95-96ca-3c38038cfffe admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:05.610874 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dbe1a4-75bb-4b95-96ca-3c38038cfffe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:05.614726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dbe1a4-75bb-4b95-96ca-3c38038cfffe admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:05.661410 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1071/2140] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:27:05 2022] POST /identity/v3/users => generated 289 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:27:05.664807 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51108b7-b71c-49e8-aa50-0d2ab34d6677 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:05.684315 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51108b7-b71c-49e8-aa50-0d2ab34d6677 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:05.693179 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b33e6f-49ee-4565-95e7-2a7416534227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:05.694392 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b33e6f-49ee-4565-95e7-2a7416534227 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:05.694898 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b33e6f-49ee-4565-95e7-2a7416534227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:05.695474 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b33e6f-49ee-4565-95e7-2a7416534227 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:05.696883 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b33e6f-49ee-4565-95e7-2a7416534227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:05.700297 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b33e6f-49ee-4565-95e7-2a7416534227 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:05.732588 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1070/2141] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:27:05.735647 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dbe1a4-75bb-4b95-96ca-3c38038cfffe admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:05.759971 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dbe1a4-75bb-4b95-96ca-3c38038cfffe admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:05.768970 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4455f988-b2e2-4172-8609-8baa1e0db989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:05.770020 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455f988-b2e2-4172-8609-8baa1e0db989 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:05.770446 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455f988-b2e2-4172-8609-8baa1e0db989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:05.770876 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455f988-b2e2-4172-8609-8baa1e0db989 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:05.772484 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4455f988-b2e2-4172-8609-8baa1e0db989 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:05.776307 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4455f988-b2e2-4172-8609-8baa1e0db989 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:05.801502 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1072/2142] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:27:05.810066 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b33e6f-49ee-4565-95e7-2a7416534227 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:05.827933 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b33e6f-49ee-4565-95e7-2a7416534227 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:05.836633 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c34df7-c658-4bf9-a87d-3898074a1556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:05.837603 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c34df7-c658-4bf9-a87d-3898074a1556 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:05.837895 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c34df7-c658-4bf9-a87d-3898074a1556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:05.838182 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c34df7-c658-4bf9-a87d-3898074a1556 admin admin] PATH_INFO: `/v3/projects/ac9a533f6f6f49e2bd73771fc99919c0/users/581ae6d40d34447da6651f3b2c21bdf0/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:05.873711 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c34df7-c658-4bf9-a87d-3898074a1556 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac9a533f6f6f49e2bd73771fc99919c0, user_id=581ae6d40d34447da6651f3b2c21bdf0, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:05.879368 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c34df7-c658-4bf9-a87d-3898074a1556 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:05.907589 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1071/2143] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:27:05 2022] PUT /identity/v3/projects/ac9a533f6f6f49e2bd73771fc99919c0/users/581ae6d40d34447da6651f3b2c21bdf0/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:27:05.917121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a030d49-0de5-4dc9-8cb0-be1f6e578255 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:05.917623 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a030d49-0de5-4dc9-8cb0-be1f6e578255 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:05.918366 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a030d49-0de5-4dc9-8cb0-be1f6e578255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:05.923543 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c34df7-c658-4bf9-a87d-3898074a1556 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:05.926538 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a6c34df7-c658-4bf9-a87d-3898074a1556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:05.949948 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1a030d49-0de5-4dc9-8cb0-be1f6e578255 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:05.951470 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1a030d49-0de5-4dc9-8cb0-be1f6e578255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:06.056146 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1073/2144] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:27:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:27:06.068513 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a030d49-0de5-4dc9-8cb0-be1f6e578255 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:06.071274 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c34df7-c658-4bf9-a87d-3898074a1556 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:06.071765 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1a030d49-0de5-4dc9-8cb0-be1f6e578255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:06.087376 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a6695-f061-4005-b3b2-a1224b0c256e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:06.088622 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a6695-f061-4005-b3b2-a1224b0c256e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:06.089397 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a6695-f061-4005-b3b2-a1224b0c256e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:06.089892 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a6695-f061-4005-b3b2-a1224b0c256e admin admin] PATH_INFO: `/v3/projects/470eead9dee24f1fb0660c5bffd7ed28` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:06.092706 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a6695-f061-4005-b3b2-a1224b0c256e admin admin] RBAC: Authorizing `identity:delete_project(project_id=470eead9dee24f1fb0660c5bffd7ed28)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:06.096672 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a6695-f061-4005-b3b2-a1224b0c256e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:06.186674 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a030d49-0de5-4dc9-8cb0-be1f6e578255 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:06.205566 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-171a6695-f061-4005-b3b2-a1224b0c256e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '470eead9dee24f1fb0660c5bffd7ed28', 'request_id': 'req-171a6695-f061-4005-b3b2-a1224b0c256e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:06.213314 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:27:06.224095 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-171a6695-f061-4005-b3b2-a1224b0c256e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '470eead9dee24f1fb0660c5bffd7ed28', 'request_id': 'req-171a6695-f061-4005-b3b2-a1224b0c256e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:06.236539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-171a6695-f061-4005-b3b2-a1224b0c256e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '470eead9dee24f1fb0660c5bffd7ed28', 'request_id': 'req-171a6695-f061-4005-b3b2-a1224b0c256e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:06.247833 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:27:06.258287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-171a6695-f061-4005-b3b2-a1224b0c256e admin admin] The token cache is being invalidate because project 470eead9dee24f1fb0660c5bffd7ed28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:27:06.266908 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-171a6695-f061-4005-b3b2-a1224b0c256e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:06.266908 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-171a6695-f061-4005-b3b2-a1224b0c256e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:06.266908 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1072/2145] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:27:05 2022] DELETE /identity/v3/projects/470eead9dee24f1fb0660c5bffd7ed28 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:27:06.266908 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a6695-f061-4005-b3b2-a1224b0c256e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:06.271100 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-171a6695-f061-4005-b3b2-a1224b0c256e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:06.292986 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3852551-79c9-4774-8927-fd781bf24b71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:06.294088 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3852551-79c9-4774-8927-fd781bf24b71 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:06.294556 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3852551-79c9-4774-8927-fd781bf24b71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:06.295018 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3852551-79c9-4774-8927-fd781bf24b71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:06.298374 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b3852551-79c9-4774-8927-fd781bf24b71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:06.390128 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a6695-f061-4005-b3b2-a1224b0c256e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:06.395888 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4937dc79-2644-4c5d-8d4c-dc908e3a9d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:06.397902 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937dc79-2644-4c5d-8d4c-dc908e3a9d28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:06.398609 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937dc79-2644-4c5d-8d4c-dc908e3a9d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:06.399027 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937dc79-2644-4c5d-8d4c-dc908e3a9d28 admin admin] PATH_INFO: `/v3/users/3f0b2bfc8a9141388f13f0cf8be86791` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:06.400685 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4937dc79-2644-4c5d-8d4c-dc908e3a9d28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f0b2bfc8a9141388f13f0cf8be86791)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:06.402815 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4937dc79-2644-4c5d-8d4c-dc908e3a9d28 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:06.415924 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3852551-79c9-4774-8927-fd781bf24b71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:06.417811 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3852551-79c9-4774-8927-fd781bf24b71 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:06.431392 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1074/2146] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:27:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:27:06.432547 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3852551-79c9-4774-8927-fd781bf24b71 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:06.434290 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b3852551-79c9-4774-8927-fd781bf24b71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:06.456282 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3852551-79c9-4774-8927-fd781bf24b71 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:06.465170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843c3ad-de82-4658-a825-bbcee80ee065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:06.466103 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843c3ad-de82-4658-a825-bbcee80ee065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:06.466519 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843c3ad-de82-4658-a825-bbcee80ee065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:06.466936 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843c3ad-de82-4658-a825-bbcee80ee065 admin admin] PATH_INFO: `/v3/users/6aa176ba6788419c87001ac6589dad28` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:06.469957 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8843c3ad-de82-4658-a825-bbcee80ee065 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6aa176ba6788419c87001ac6589dad28)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:06.473455 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8843c3ad-de82-4658-a825-bbcee80ee065 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:06.487408 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4937dc79-2644-4c5d-8d4c-dc908e3a9d28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f0b2bfc8a9141388f13f0cf8be86791'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:06.500007 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4937dc79-2644-4c5d-8d4c-dc908e3a9d28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f0b2bfc8a9141388f13f0cf8be86791'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:06.523713 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4937dc79-2644-4c5d-8d4c-dc908e3a9d28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f0b2bfc8a9141388f13f0cf8be86791'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:06.525349 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4937dc79-2644-4c5d-8d4c-dc908e3a9d28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f0b2bfc8a9141388f13f0cf8be86791'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:06.550427 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4937dc79-2644-4c5d-8d4c-dc908e3a9d28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f0b2bfc8a9141388f13f0cf8be86791'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:06.559431 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1073/2147] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:27:06 2022] DELETE /identity/v3/users/3f0b2bfc8a9141388f13f0cf8be86791 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:27:06.565993 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bdc055-9e90-47d5-ab23-54e375ee1e5b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:06.566389 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bdc055-9e90-47d5-ab23-54e375ee1e5b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:06.566716 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bdc055-9e90-47d5-ab23-54e375ee1e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:06.573883 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8843c3ad-de82-4658-a825-bbcee80ee065 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6aa176ba6788419c87001ac6589dad28'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:06.593997 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8843c3ad-de82-4658-a825-bbcee80ee065 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6aa176ba6788419c87001ac6589dad28'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:06.608962 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e3bdc055-9e90-47d5-ab23-54e375ee1e5b None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:06.609382 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8843c3ad-de82-4658-a825-bbcee80ee065 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6aa176ba6788419c87001ac6589dad28'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:06.610302 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8843c3ad-de82-4658-a825-bbcee80ee065 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6aa176ba6788419c87001ac6589dad28'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:06.611817 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e3bdc055-9e90-47d5-ab23-54e375ee1e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:06.613172 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8843c3ad-de82-4658-a825-bbcee80ee065 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6aa176ba6788419c87001ac6589dad28'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:06.668077 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1075/2148] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:27:06 2022] DELETE /identity/v3/users/6aa176ba6788419c87001ac6589dad28 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:27:06.771212 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1074/2149] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:27:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:27:06.826744 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843c3ad-de82-4658-a825-bbcee80ee065 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:06.829646 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8843c3ad-de82-4658-a825-bbcee80ee065 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:06.890541 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9dc106-60d1-4cb7-95b0-07f6d8f4eb9f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:06.891104 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9dc106-60d1-4cb7-95b0-07f6d8f4eb9f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:06.891572 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9dc106-60d1-4cb7-95b0-07f6d8f4eb9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:06.955207 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4e9dc106-60d1-4cb7-95b0-07f6d8f4eb9f None None] MFA Rules not processed for user `dc63fdeacd514eac8e4a5882190f8d66`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:06.957371 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4e9dc106-60d1-4cb7-95b0-07f6d8f4eb9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:06.969499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:27:07.012687 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:27:07.013021 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843c3ad-de82-4658-a825-bbcee80ee065 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:07.019771 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a044dd7-a8ba-4b28-ae12-5bfe7639a676 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:07.022624 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a044dd7-a8ba-4b28-ae12-5bfe7639a676 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:07.022926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a044dd7-a8ba-4b28-ae12-5bfe7639a676 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:07.023218 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a044dd7-a8ba-4b28-ae12-5bfe7639a676 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:07.192705 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1075/2150] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:27:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:27:07.201477 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a044dd7-a8ba-4b28-ae12-5bfe7639a676 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:07.204541 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a044dd7-a8ba-4b28-ae12-5bfe7639a676 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:07.222121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1076/2151] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:27:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:27:07.618596 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9dc106-60d1-4cb7-95b0-07f6d8f4eb9f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:07.621019 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4e9dc106-60d1-4cb7-95b0-07f6d8f4eb9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:07.654336 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9dc106-60d1-4cb7-95b0-07f6d8f4eb9f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:07.665217 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613b8e9-0db2-4d24-859b-7ac08b4134e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:07.666829 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613b8e9-0db2-4d24-859b-7ac08b4134e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:07.667686 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613b8e9-0db2-4d24-859b-7ac08b4134e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:07.668282 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613b8e9-0db2-4d24-859b-7ac08b4134e4 admin admin] PATH_INFO: `/v3/projects/9a1617ab5e86475db7b124f52b488e5d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:07.671269 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d613b8e9-0db2-4d24-859b-7ac08b4134e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a1617ab5e86475db7b124f52b488e5d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:07.675235 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d613b8e9-0db2-4d24-859b-7ac08b4134e4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:07.824468 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d613b8e9-0db2-4d24-859b-7ac08b4134e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a1617ab5e86475db7b124f52b488e5d', 'request_id': 'req-d613b8e9-0db2-4d24-859b-7ac08b4134e4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:07.838158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d613b8e9-0db2-4d24-859b-7ac08b4134e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a1617ab5e86475db7b124f52b488e5d', 'request_id': 'req-d613b8e9-0db2-4d24-859b-7ac08b4134e4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:07.853496 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d613b8e9-0db2-4d24-859b-7ac08b4134e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a1617ab5e86475db7b124f52b488e5d', 'request_id': 'req-d613b8e9-0db2-4d24-859b-7ac08b4134e4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:07.873693 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d613b8e9-0db2-4d24-859b-7ac08b4134e4 admin admin] The token cache is being invalidate because project 9a1617ab5e86475db7b124f52b488e5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:27:07.874202 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d613b8e9-0db2-4d24-859b-7ac08b4134e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:07.875224 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d613b8e9-0db2-4d24-859b-7ac08b4134e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:07.877819 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1076/2152] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:27:07 2022] DELETE /identity/v3/projects/9a1617ab5e86475db7b124f52b488e5d => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:27:07.886040 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a044dd7-a8ba-4b28-ae12-5bfe7639a676 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:07.908111 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a044dd7-a8ba-4b28-ae12-5bfe7639a676 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:07.957957 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613b8e9-0db2-4d24-859b-7ac08b4134e4 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:07.961301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d613b8e9-0db2-4d24-859b-7ac08b4134e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:08.073428 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e112c9f-903a-4046-a4aa-2a479c521619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:08.075997 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e112c9f-903a-4046-a4aa-2a479c521619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:08.076526 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e112c9f-903a-4046-a4aa-2a479c521619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:08.077041 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e112c9f-903a-4046-a4aa-2a479c521619 admin admin] PATH_INFO: `/v3/users/69b4c6c09c4741939b13309ccbe9783f` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:08.080928 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e112c9f-903a-4046-a4aa-2a479c521619 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69b4c6c09c4741939b13309ccbe9783f)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:08.086008 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e112c9f-903a-4046-a4aa-2a479c521619 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:08.119321 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613b8e9-0db2-4d24-859b-7ac08b4134e4 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:08.132071 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332e7e0-63c9-4642-980d-17f7eb1ddf2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:08.135564 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332e7e0-63c9-4642-980d-17f7eb1ddf2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:08.136111 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332e7e0-63c9-4642-980d-17f7eb1ddf2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:08.137102 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332e7e0-63c9-4642-980d-17f7eb1ddf2f admin admin] PATH_INFO: `/v3/projects/a84bdd938566421f9f98910d89456e09` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:08.142283 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332e7e0-63c9-4642-980d-17f7eb1ddf2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a84bdd938566421f9f98910d89456e09)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:08.144960 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332e7e0-63c9-4642-980d-17f7eb1ddf2f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:08.217162 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8e112c9f-903a-4046-a4aa-2a479c521619 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69b4c6c09c4741939b13309ccbe9783f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:08.248110 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8e112c9f-903a-4046-a4aa-2a479c521619 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69b4c6c09c4741939b13309ccbe9783f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:08.265973 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7332e7e0-63c9-4642-980d-17f7eb1ddf2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a84bdd938566421f9f98910d89456e09', 'request_id': 'req-7332e7e0-63c9-4642-980d-17f7eb1ddf2f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:08.272195 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:27:08.278151 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8e112c9f-903a-4046-a4aa-2a479c521619 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69b4c6c09c4741939b13309ccbe9783f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:08.279149 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8e112c9f-903a-4046-a4aa-2a479c521619 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69b4c6c09c4741939b13309ccbe9783f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:08.279956 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8e112c9f-903a-4046-a4aa-2a479c521619 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69b4c6c09c4741939b13309ccbe9783f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:08.289167 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7332e7e0-63c9-4642-980d-17f7eb1ddf2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a84bdd938566421f9f98910d89456e09', 'request_id': 'req-7332e7e0-63c9-4642-980d-17f7eb1ddf2f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:08.301017 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7332e7e0-63c9-4642-980d-17f7eb1ddf2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a84bdd938566421f9f98910d89456e09', 'request_id': 'req-7332e7e0-63c9-4642-980d-17f7eb1ddf2f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:08.325046 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7332e7e0-63c9-4642-980d-17f7eb1ddf2f admin admin] The token cache is being invalidate because project a84bdd938566421f9f98910d89456e09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:27:08.326114 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7332e7e0-63c9-4642-980d-17f7eb1ddf2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:08.328142 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7332e7e0-63c9-4642-980d-17f7eb1ddf2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:08.332389 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1077/2153] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:27:07 2022] DELETE /identity/v3/projects/a84bdd938566421f9f98910d89456e09 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:27:08.343064 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa01995-f078-43bf-ad86-6f704c8d5221 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:08.344222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa01995-f078-43bf-ad86-6f704c8d5221 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:08.345355 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa01995-f078-43bf-ad86-6f704c8d5221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:08.357917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1077/2154] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:27:07 2022] DELETE /identity/v3/users/69b4c6c09c4741939b13309ccbe9783f => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:27:08.394461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2fa01995-f078-43bf-ad86-6f704c8d5221 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:08.399079 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2fa01995-f078-43bf-ad86-6f704c8d5221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:08.663943 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1078/2155] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:27:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:27:08.674860 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f27831-96f4-40c7-a997-4de213af41da None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:08.675382 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f27831-96f4-40c7-a997-4de213af41da None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:08.676417 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f27831-96f4-40c7-a997-4de213af41da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:08.727435 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-35f27831-96f4-40c7-a997-4de213af41da None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:08.728913 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-35f27831-96f4-40c7-a997-4de213af41da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:08.745567 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1078/2156] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:27:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:27:08.747541 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa01995-f078-43bf-ad86-6f704c8d5221 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:08.762312 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa01995-f078-43bf-ad86-6f704c8d5221 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:08.770884 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8f821-f1c3-4b50-a553-9cc24c39c32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:08.772293 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8f821-f1c3-4b50-a553-9cc24c39c32b admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:08.772961 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8f821-f1c3-4b50-a553-9cc24c39c32b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:08.773865 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8f821-f1c3-4b50-a553-9cc24c39c32b admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:08.774962 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa8f821-f1c3-4b50-a553-9cc24c39c32b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:27:08.775911 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa8f821-f1c3-4b50-a553-9cc24c39c32b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:08.778537 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa8f821-f1c3-4b50-a553-9cc24c39c32b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:08.813469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1079/2157] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 19:27:08.821445 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f27831-96f4-40c7-a997-4de213af41da None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:08.837821 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:27:08.867219 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f27831-96f4-40c7-a997-4de213af41da None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:08.909201 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a37346a-3a65-40fc-93bb-7bc41f5f906b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:08.911900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a37346a-3a65-40fc-93bb-7bc41f5f906b admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:08.913490 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a37346a-3a65-40fc-93bb-7bc41f5f906b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:08.915705 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a37346a-3a65-40fc-93bb-7bc41f5f906b admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:08.918124 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a37346a-3a65-40fc-93bb-7bc41f5f906b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:08.927060 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a37346a-3a65-40fc-93bb-7bc41f5f906b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:09.003703 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8f821-f1c3-4b50-a553-9cc24c39c32b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:09.012184 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1079/2158] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:27:08 2022] POST /identity/v3/projects => generated 383 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:27:09.024245 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8f821-f1c3-4b50-a553-9cc24c39c32b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:09.249207 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf3ca5-4df8-44b4-a117-24134cd1b18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:09.250102 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf3ca5-4df8-44b4-a117-24134cd1b18a admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:09.250268 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf3ca5-4df8-44b4-a117-24134cd1b18a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:09.250586 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf3ca5-4df8-44b4-a117-24134cd1b18a admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:09.252197 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf3ca5-4df8-44b4-a117-24134cd1b18a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:09.268707 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf3ca5-4df8-44b4-a117-24134cd1b18a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:09.359631 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1080/2159] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:27:09 2022] POST /identity/v3/users => generated 292 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:27:09.361662 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a37346a-3a65-40fc-93bb-7bc41f5f906b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:09.389970 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a37346a-3a65-40fc-93bb-7bc41f5f906b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:09.418874 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e1b489-3fe0-4d25-b06e-feec944bde89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:09.420213 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1b489-3fe0-4d25-b06e-feec944bde89 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:09.420757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1b489-3fe0-4d25-b06e-feec944bde89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:09.421300 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf3ca5-4df8-44b4-a117-24134cd1b18a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:09.421869 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1b489-3fe0-4d25-b06e-feec944bde89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:09.425569 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e1b489-3fe0-4d25-b06e-feec944bde89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:09.425890 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fcaf3ca5-4df8-44b4-a117-24134cd1b18a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:09.430043 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e1b489-3fe0-4d25-b06e-feec944bde89 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:09.451044 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf3ca5-4df8-44b4-a117-24134cd1b18a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:09.471433 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53357a8-08f1-4aa9-8fb7-0d2464869bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:09.473160 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1080/2160] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:27:09.475321 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e1b489-3fe0-4d25-b06e-feec944bde89 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:09.476879 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53357a8-08f1-4aa9-8fb7-0d2464869bc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:09.477816 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53357a8-08f1-4aa9-8fb7-0d2464869bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:09.478258 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53357a8-08f1-4aa9-8fb7-0d2464869bc7 admin admin] PATH_INFO: `/v3/projects/181944174a4d4d1cb511f28c774de320` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:09.481317 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53357a8-08f1-4aa9-8fb7-0d2464869bc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=181944174a4d4d1cb511f28c774de320)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:09.487144 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53357a8-08f1-4aa9-8fb7-0d2464869bc7 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:09.498002 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e1b489-3fe0-4d25-b06e-feec944bde89 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:09.506633 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f52d91-8d95-43ba-9ca9-e7871769e260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:09.507829 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f52d91-8d95-43ba-9ca9-e7871769e260 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:09.508300 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f52d91-8d95-43ba-9ca9-e7871769e260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:09.508763 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f52d91-8d95-43ba-9ca9-e7871769e260 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:09.510126 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f52d91-8d95-43ba-9ca9-e7871769e260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:09.512867 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f52d91-8d95-43ba-9ca9-e7871769e260 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:09.540626 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1081/2161] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:27:09.542117 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f52d91-8d95-43ba-9ca9-e7871769e260 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:09.563941 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f52d91-8d95-43ba-9ca9-e7871769e260 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:09.574349 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac007a-aca6-4a65-86fe-51239184d677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:09.575917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac007a-aca6-4a65-86fe-51239184d677 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:09.576378 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac007a-aca6-4a65-86fe-51239184d677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:09.576838 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac007a-aca6-4a65-86fe-51239184d677 admin admin] PATH_INFO: `/v3/projects/43cfdd0ba7de4e46a17d4c0d698159cf/users/324738dd872e445ba845a101f45fa595/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:09.610411 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac007a-aca6-4a65-86fe-51239184d677 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43cfdd0ba7de4e46a17d4c0d698159cf, user_id=324738dd872e445ba845a101f45fa595, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:09.618553 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac007a-aca6-4a65-86fe-51239184d677 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:09.669005 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:27:09.674475 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b53357a8-08f1-4aa9-8fb7-0d2464869bc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '181944174a4d4d1cb511f28c774de320', 'request_id': 'req-b53357a8-08f1-4aa9-8fb7-0d2464869bc7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:09.680308 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1082/2162] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:27:09 2022] PUT /identity/v3/projects/43cfdd0ba7de4e46a17d4c0d698159cf/users/324738dd872e445ba845a101f45fa595/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:27:09.691361 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d667d-4a25-42bd-96c0-1443bc40469e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:09.696365 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d667d-4a25-42bd-96c0-1443bc40469e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:09.697464 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d667d-4a25-42bd-96c0-1443bc40469e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:09.711560 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b53357a8-08f1-4aa9-8fb7-0d2464869bc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '181944174a4d4d1cb511f28c774de320', 'request_id': 'req-b53357a8-08f1-4aa9-8fb7-0d2464869bc7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:09.730106 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b53357a8-08f1-4aa9-8fb7-0d2464869bc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '181944174a4d4d1cb511f28c774de320', 'request_id': 'req-b53357a8-08f1-4aa9-8fb7-0d2464869bc7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:09.743053 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-cd1d667d-4a25-42bd-96c0-1443bc40469e None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:09.745624 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cd1d667d-4a25-42bd-96c0-1443bc40469e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:09.769212 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b53357a8-08f1-4aa9-8fb7-0d2464869bc7 admin admin] The token cache is being invalidate because project 181944174a4d4d1cb511f28c774de320 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:27:09.773110 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b53357a8-08f1-4aa9-8fb7-0d2464869bc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:09.775888 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b53357a8-08f1-4aa9-8fb7-0d2464869bc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:09.779295 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1081/2163] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:27:09 2022] DELETE /identity/v3/projects/181944174a4d4d1cb511f28c774de320 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:27:09.790521 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9597415-6273-4056-bd88-692abed16298 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:09.791084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9597415-6273-4056-bd88-692abed16298 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:09.791482 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9597415-6273-4056-bd88-692abed16298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:09.843304 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d9597415-6273-4056-bd88-692abed16298 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:09.844971 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d9597415-6273-4056-bd88-692abed16298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:09.942367 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1083/2164] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:27:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:27:09.953136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d667d-4a25-42bd-96c0-1443bc40469e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:09.954695 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1082/2165] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:27:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:27:09.964989 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43671693-422a-459b-be25-67e32724c915 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:09.965282 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cd1d667d-4a25-42bd-96c0-1443bc40469e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:09.965427 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43671693-422a-459b-be25-67e32724c915 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:09.968362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43671693-422a-459b-be25-67e32724c915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:10.029754 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-43671693-422a-459b-be25-67e32724c915 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:10.038971 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-43671693-422a-459b-be25-67e32724c915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:10.076030 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1083/2166] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:27:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:27:10.078083 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43671693-422a-459b-be25-67e32724c915 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:10.104882 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43671693-422a-459b-be25-67e32724c915 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:10.121063 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b4de91-123f-4d22-b44d-c43e69d546fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:10.122299 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4de91-123f-4d22-b44d-c43e69d546fa admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:10.123741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4de91-123f-4d22-b44d-c43e69d546fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:10.126588 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4de91-123f-4d22-b44d-c43e69d546fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:10.128270 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b4de91-123f-4d22-b44d-c43e69d546fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:27:10.129333 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b4de91-123f-4d22-b44d-c43e69d546fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:10.136111 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b4de91-123f-4d22-b44d-c43e69d546fa admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:10.182112 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1084/2167] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:27:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:27:10.186837 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d667d-4a25-42bd-96c0-1443bc40469e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:10.194459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b4de91-123f-4d22-b44d-c43e69d546fa admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:10.197527 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483c1145-cefd-4762-ad1c-d293d057b153 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:10.198578 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c1145-cefd-4762-ad1c-d293d057b153 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:10.203128 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c1145-cefd-4762-ad1c-d293d057b153 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:10.203740 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c1145-cefd-4762-ad1c-d293d057b153 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:10.229518 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b4de91-123f-4d22-b44d-c43e69d546fa admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:10.239725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c1628-07c5-4703-b671-bffcb2870d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:10.242743 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c1628-07c5-4703-b671-bffcb2870d23 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:10.243459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c1628-07c5-4703-b671-bffcb2870d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:10.243984 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c1628-07c5-4703-b671-bffcb2870d23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:10.245391 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528c1628-07c5-4703-b671-bffcb2870d23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:10.249004 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483c1145-cefd-4762-ad1c-d293d057b153 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:10.249260 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528c1628-07c5-4703-b671-bffcb2870d23 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:10.251653 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483c1145-cefd-4762-ad1c-d293d057b153 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:10.294727 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1084/2168] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:27:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:27:10.345221 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1085/2169] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:27:10 2022] POST /identity/v3/projects => generated 381 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:27:10.351241 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483c1145-cefd-4762-ad1c-d293d057b153 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:10.388573 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483c1145-cefd-4762-ad1c-d293d057b153 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:10.536031 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:27:10.607669 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4edfef-6def-49fd-8092-4bfe4ad94ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:10.608724 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4edfef-6def-49fd-8092-4bfe4ad94ac5 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:10.609179 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4edfef-6def-49fd-8092-4bfe4ad94ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:10.609678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4edfef-6def-49fd-8092-4bfe4ad94ac5 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:10.611098 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4edfef-6def-49fd-8092-4bfe4ad94ac5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:10.617013 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4edfef-6def-49fd-8092-4bfe4ad94ac5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:10.667190 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1085/2170] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:27:10 2022] POST /identity/v3/users => generated 291 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:27:10.673176 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c1628-07c5-4703-b671-bffcb2870d23 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:10.702207 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c1628-07c5-4703-b671-bffcb2870d23 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:10.742107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8080b72-a127-40fd-b55a-61fb3c1ae4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:10.749239 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8080b72-a127-40fd-b55a-61fb3c1ae4fe admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:10.749760 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8080b72-a127-40fd-b55a-61fb3c1ae4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:10.751736 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8080b72-a127-40fd-b55a-61fb3c1ae4fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:10.761764 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8080b72-a127-40fd-b55a-61fb3c1ae4fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:10.770311 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8080b72-a127-40fd-b55a-61fb3c1ae4fe admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:10.792114 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1086/2171] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:27:10.799292 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4edfef-6def-49fd-8092-4bfe4ad94ac5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:10.825564 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4edfef-6def-49fd-8092-4bfe4ad94ac5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:10.843989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc6fd7-eb6d-421f-b43a-9e4b498cb763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:10.845119 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc6fd7-eb6d-421f-b43a-9e4b498cb763 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:10.845579 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc6fd7-eb6d-421f-b43a-9e4b498cb763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:10.846117 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc6fd7-eb6d-421f-b43a-9e4b498cb763 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:10.847322 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc6fd7-eb6d-421f-b43a-9e4b498cb763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:10.852148 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc6fd7-eb6d-421f-b43a-9e4b498cb763 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:10.884951 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1086/2172] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:27:10.886232 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8080b72-a127-40fd-b55a-61fb3c1ae4fe admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:10.911422 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8080b72-a127-40fd-b55a-61fb3c1ae4fe admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:10.933118 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb298f40-5c89-40f1-bb91-f641a49da2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:10.934958 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb298f40-5c89-40f1-bb91-f641a49da2b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:10.935398 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb298f40-5c89-40f1-bb91-f641a49da2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:10.935994 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb298f40-5c89-40f1-bb91-f641a49da2b0 admin admin] PATH_INFO: `/v3/projects/77d870baf2fa44aab63e3337fdf07e7f/users/b0a1b7a33a5a450ab09bc223ce2930a2/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:11.014784 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb298f40-5c89-40f1-bb91-f641a49da2b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77d870baf2fa44aab63e3337fdf07e7f, user_id=b0a1b7a33a5a450ab09bc223ce2930a2, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:11.025476 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb298f40-5c89-40f1-bb91-f641a49da2b0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:11.090716 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1087/2173] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:27:10 2022] PUT /identity/v3/projects/77d870baf2fa44aab63e3337fdf07e7f/users/b0a1b7a33a5a450ab09bc223ce2930a2/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:27:11.114911 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52101e7f-e2ff-40b7-a538-a644b5dd8d04 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:11.117250 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52101e7f-e2ff-40b7-a538-a644b5dd8d04 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:11.117794 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52101e7f-e2ff-40b7-a538-a644b5dd8d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:11.214902 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-52101e7f-e2ff-40b7-a538-a644b5dd8d04 None None] MFA Rules not processed for user `b0a1b7a33a5a450ab09bc223ce2930a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:11.217442 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-52101e7f-e2ff-40b7-a538-a644b5dd8d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:11.486692 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb298f40-5c89-40f1-bb91-f641a49da2b0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:11.487173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1087/2174] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:27:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:27:11.737449 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb298f40-5c89-40f1-bb91-f641a49da2b0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:11.760599 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f67cae-d4c7-4fd0-bb51-859d36c7f5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:11.761559 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f67cae-d4c7-4fd0-bb51-859d36c7f5e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:11.761960 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f67cae-d4c7-4fd0-bb51-859d36c7f5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:11.762473 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f67cae-d4c7-4fd0-bb51-859d36c7f5e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:11.763467 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f67cae-d4c7-4fd0-bb51-859d36c7f5e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:11.766604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f67cae-d4c7-4fd0-bb51-859d36c7f5e0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:11.777605 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:27:11.853123 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1088/2175] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:27:11 2022] POST /identity/v3/projects => generated 379 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:27:11.857502 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52101e7f-e2ff-40b7-a538-a644b5dd8d04 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:11.884659 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52101e7f-e2ff-40b7-a538-a644b5dd8d04 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:12.097224 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d108a-6da3-400b-88c0-3e178baf67b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:12.098616 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d108a-6da3-400b-88c0-3e178baf67b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:12.099263 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d108a-6da3-400b-88c0-3e178baf67b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:12.099871 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d108a-6da3-400b-88c0-3e178baf67b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:12.101340 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d108a-6da3-400b-88c0-3e178baf67b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:12.109727 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d108a-6da3-400b-88c0-3e178baf67b5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:12.180812 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1088/2176] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:27:11 2022] POST /identity/v3/users => generated 296 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:27:12.182571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f67cae-d4c7-4fd0-bb51-859d36c7f5e0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:12.218188 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f67cae-d4c7-4fd0-bb51-859d36c7f5e0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:12.242726 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb8ec1-dfa4-4806-bc07-8bf48ad0bc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:12.244187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb8ec1-dfa4-4806-bc07-8bf48ad0bc45 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:12.244797 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb8ec1-dfa4-4806-bc07-8bf48ad0bc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:12.245628 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb8ec1-dfa4-4806-bc07-8bf48ad0bc45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:12.247133 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb8ec1-dfa4-4806-bc07-8bf48ad0bc45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:12.253491 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb8ec1-dfa4-4806-bc07-8bf48ad0bc45 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:12.302443 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1089/2177] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:27:12.305787 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d108a-6da3-400b-88c0-3e178baf67b5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:12.338526 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d108a-6da3-400b-88c0-3e178baf67b5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:12.361980 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaffd86d-204d-45e7-86ff-734fab527ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:12.365113 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaffd86d-204d-45e7-86ff-734fab527ab2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:12.365915 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaffd86d-204d-45e7-86ff-734fab527ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:12.367209 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaffd86d-204d-45e7-86ff-734fab527ab2 admin admin] PATH_INFO: `/v3/projects/c44d7520fb7e456dbeadc809570083b6/users/a9a5c13560684ed288aed52bad065e9c/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:12.423266 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaffd86d-204d-45e7-86ff-734fab527ab2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c44d7520fb7e456dbeadc809570083b6, user_id=a9a5c13560684ed288aed52bad065e9c, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:12.450196 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaffd86d-204d-45e7-86ff-734fab527ab2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:12.499396 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1089/2178] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:27:12 2022] PUT /identity/v3/projects/c44d7520fb7e456dbeadc809570083b6/users/a9a5c13560684ed288aed52bad065e9c/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:27:12.506769 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb8ec1-dfa4-4806-bc07-8bf48ad0bc45 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:12.525679 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb8ec1-dfa4-4806-bc07-8bf48ad0bc45 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:12.645635 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65971fd2-6b81-4913-bba6-238cb0c86f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:12.646833 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65971fd2-6b81-4913-bba6-238cb0c86f8c admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:12.647465 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65971fd2-6b81-4913-bba6-238cb0c86f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:12.647884 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65971fd2-6b81-4913-bba6-238cb0c86f8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:12.648765 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65971fd2-6b81-4913-bba6-238cb0c86f8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:12.650957 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65971fd2-6b81-4913-bba6-238cb0c86f8c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:12.667340 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1090/2179] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:27:12.674642 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaffd86d-204d-45e7-86ff-734fab527ab2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:12.706186 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaffd86d-204d-45e7-86ff-734fab527ab2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:12.715899 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14248d9d-3764-4caa-ab71-f7188a8ca99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:12.718073 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14248d9d-3764-4caa-ab71-f7188a8ca99f admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:12.718932 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14248d9d-3764-4caa-ab71-f7188a8ca99f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:12.720215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14248d9d-3764-4caa-ab71-f7188a8ca99f admin admin] PATH_INFO: `/v3/projects/c44d7520fb7e456dbeadc809570083b6/users/a9a5c13560684ed288aed52bad065e9c/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:12.724781 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14248d9d-3764-4caa-ab71-f7188a8ca99f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c44d7520fb7e456dbeadc809570083b6, user_id=a9a5c13560684ed288aed52bad065e9c, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:12.731260 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14248d9d-3764-4caa-ab71-f7188a8ca99f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:12.752484 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:27:12.761846 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1090/2180] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:27:12 2022] PUT /identity/v3/projects/c44d7520fb7e456dbeadc809570083b6/users/a9a5c13560684ed288aed52bad065e9c/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:27:12.782470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb2072-01ad-47f3-b90b-d5d0b5aee877 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:12.782994 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb2072-01ad-47f3-b90b-d5d0b5aee877 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:12.783628 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb2072-01ad-47f3-b90b-d5d0b5aee877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:12.882346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5acb2072-01ad-47f3-b90b-d5d0b5aee877 None None] MFA Rules not processed for user `a9a5c13560684ed288aed52bad065e9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:12.886558 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5acb2072-01ad-47f3-b90b-d5d0b5aee877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:13.150375 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1091/2181] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:27:12 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:27:13.173620 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14248d9d-3764-4caa-ab71-f7188a8ca99f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:13.180303 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-14248d9d-3764-4caa-ab71-f7188a8ca99f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:13.313856 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14248d9d-3764-4caa-ab71-f7188a8ca99f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:13.334671 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abadf6d-36c8-4bd1-878f-245c1d583c42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:13.335849 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abadf6d-36c8-4bd1-878f-245c1d583c42 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:13.336346 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abadf6d-36c8-4bd1-878f-245c1d583c42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:13.336819 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abadf6d-36c8-4bd1-878f-245c1d583c42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:13.359956 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abadf6d-36c8-4bd1-878f-245c1d583c42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:13.362664 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abadf6d-36c8-4bd1-878f-245c1d583c42 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:13.380388 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1091/2182] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:27:13 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:27:14.627994 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acb2072-01ad-47f3-b90b-d5d0b5aee877 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:14.917605 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:27:14.946988 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acb2072-01ad-47f3-b90b-d5d0b5aee877 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:14.979412 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd086ff4-7bc6-47a1-a4ae-e051cd17b7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:14.981183 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd086ff4-7bc6-47a1-a4ae-e051cd17b7ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:14.981789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd086ff4-7bc6-47a1-a4ae-e051cd17b7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:14.982420 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd086ff4-7bc6-47a1-a4ae-e051cd17b7ab admin admin] PATH_INFO: `/v3/users/b0a1b7a33a5a450ab09bc223ce2930a2` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:14.993782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd086ff4-7bc6-47a1-a4ae-e051cd17b7ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0a1b7a33a5a450ab09bc223ce2930a2)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:14.999741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd086ff4-7bc6-47a1-a4ae-e051cd17b7ab admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:15.199608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cd086ff4-7bc6-47a1-a4ae-e051cd17b7ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0a1b7a33a5a450ab09bc223ce2930a2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:15.228386 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cd086ff4-7bc6-47a1-a4ae-e051cd17b7ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0a1b7a33a5a450ab09bc223ce2930a2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:15.234932 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4589d12-9165-4042-9a29-e0bea43696fe None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:15.235740 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4589d12-9165-4042-9a29-e0bea43696fe None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:15.235945 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4589d12-9165-4042-9a29-e0bea43696fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:15.270652 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cd086ff4-7bc6-47a1-a4ae-e051cd17b7ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0a1b7a33a5a450ab09bc223ce2930a2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:15.273536 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cd086ff4-7bc6-47a1-a4ae-e051cd17b7ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0a1b7a33a5a450ab09bc223ce2930a2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:15.300987 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f4589d12-9165-4042-9a29-e0bea43696fe None None] MFA Rules not processed for user `324738dd872e445ba845a101f45fa595`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:15.303848 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f4589d12-9165-4042-9a29-e0bea43696fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:15.350884 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cd086ff4-7bc6-47a1-a4ae-e051cd17b7ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0a1b7a33a5a450ab09bc223ce2930a2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:15.373888 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1092/2183] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:27:14 2022] DELETE /identity/v3/users/b0a1b7a33a5a450ab09bc223ce2930a2 => generated 0 bytes in 748 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:27:15.379697 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8e740-3b34-4e32-8013-44ad5fc749d7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:15.380073 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8e740-3b34-4e32-8013-44ad5fc749d7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:15.380508 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8e740-3b34-4e32-8013-44ad5fc749d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:15.438781 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-65b8e740-3b34-4e32-8013-44ad5fc749d7 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:15.447214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-65b8e740-3b34-4e32-8013-44ad5fc749d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:15.635677 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1092/2184] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:27:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 402 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:27:15.658702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4589d12-9165-4042-9a29-e0bea43696fe None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:15.661288 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f4589d12-9165-4042-9a29-e0bea43696fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:15.714300 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4589d12-9165-4042-9a29-e0bea43696fe None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:15.775069 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1093/2185] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:27:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:27:15.859301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8e740-3b34-4e32-8013-44ad5fc749d7 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:15.862231 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-65b8e740-3b34-4e32-8013-44ad5fc749d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:15.918003 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093caf0c-67a3-42c5-82f4-72fa57bd113c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:15.919673 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093caf0c-67a3-42c5-82f4-72fa57bd113c service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:15.920222 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093caf0c-67a3-42c5-82f4-72fa57bd113c service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:15.920780 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093caf0c-67a3-42c5-82f4-72fa57bd113c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:16.116509 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:27:16.138587 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8e740-3b34-4e32-8013-44ad5fc749d7 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:16.159775 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44eee56-1b89-47b0-8526-b827ffcf4206 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:16.161456 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44eee56-1b89-47b0-8526-b827ffcf4206 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:16.164359 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44eee56-1b89-47b0-8526-b827ffcf4206 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:16.167781 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44eee56-1b89-47b0-8526-b827ffcf4206 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:16.178281 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093caf0c-67a3-42c5-82f4-72fa57bd113c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:16.185726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093caf0c-67a3-42c5-82f4-72fa57bd113c service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:16.210494 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44eee56-1b89-47b0-8526-b827ffcf4206 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:16.213911 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44eee56-1b89-47b0-8526-b827ffcf4206 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:16.252198 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1094/2186] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:27:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:27:16.304262 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1093/2187] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:27:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 647 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:27:16.999013 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a81e4-7135-4ea1-a955-724e35262adf None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:16.999988 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a81e4-7135-4ea1-a955-724e35262adf None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:17.000591 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a81e4-7135-4ea1-a955-724e35262adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:17.192270 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-820a81e4-7135-4ea1-a955-724e35262adf None None] MFA Rules not processed for user `581ae6d40d34447da6651f3b2c21bdf0`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:17.196037 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-820a81e4-7135-4ea1-a955-724e35262adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:17.225981 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:27:17.498364 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1095/2188] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:27:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 502 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:27:17.658093 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093caf0c-67a3-42c5-82f4-72fa57bd113c service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:17.663225 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-093caf0c-67a3-42c5-82f4-72fa57bd113c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:17.694277 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093caf0c-67a3-42c5-82f4-72fa57bd113c service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:17.700807 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733aec4-1347-40fb-9a2b-4ee14301b721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:17.702048 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733aec4-1347-40fb-9a2b-4ee14301b721 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:17.702931 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733aec4-1347-40fb-9a2b-4ee14301b721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:17.703630 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733aec4-1347-40fb-9a2b-4ee14301b721 admin admin] PATH_INFO: `/v3/projects/77d870baf2fa44aab63e3337fdf07e7f` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:17.705477 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9733aec4-1347-40fb-9a2b-4ee14301b721 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77d870baf2fa44aab63e3337fdf07e7f)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:17.708561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9733aec4-1347-40fb-9a2b-4ee14301b721 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:17.804920 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a81e4-7135-4ea1-a955-724e35262adf None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:17.821400 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-820a81e4-7135-4ea1-a955-724e35262adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:17.881363 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9733aec4-1347-40fb-9a2b-4ee14301b721 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77d870baf2fa44aab63e3337fdf07e7f', 'request_id': 'req-9733aec4-1347-40fb-9a2b-4ee14301b721'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:17.908506 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9733aec4-1347-40fb-9a2b-4ee14301b721 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77d870baf2fa44aab63e3337fdf07e7f', 'request_id': 'req-9733aec4-1347-40fb-9a2b-4ee14301b721'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:17.953672 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9733aec4-1347-40fb-9a2b-4ee14301b721 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77d870baf2fa44aab63e3337fdf07e7f', 'request_id': 'req-9733aec4-1347-40fb-9a2b-4ee14301b721'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:18.002630 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9733aec4-1347-40fb-9a2b-4ee14301b721 admin admin] The token cache is being invalidate because project 77d870baf2fa44aab63e3337fdf07e7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:27:18.003457 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9733aec4-1347-40fb-9a2b-4ee14301b721 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:18.018586 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9733aec4-1347-40fb-9a2b-4ee14301b721 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:18.027320 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1094/2189] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:27:17 2022] DELETE /identity/v3/projects/77d870baf2fa44aab63e3337fdf07e7f => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:27:18.029009 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733aec4-1347-40fb-9a2b-4ee14301b721 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:18.032206 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9733aec4-1347-40fb-9a2b-4ee14301b721 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:18.155298 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a81e4-7135-4ea1-a955-724e35262adf None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:18.247256 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733aec4-1347-40fb-9a2b-4ee14301b721 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:18.258684 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6debf4fc-0188-40ee-b9f8-7ea128ccc1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:18.261399 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debf4fc-0188-40ee-b9f8-7ea128ccc1d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:18.262169 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debf4fc-0188-40ee-b9f8-7ea128ccc1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:18.262844 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debf4fc-0188-40ee-b9f8-7ea128ccc1d5 admin admin] PATH_INFO: `/v3/users/a9a5c13560684ed288aed52bad065e9c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:18.267859 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6debf4fc-0188-40ee-b9f8-7ea128ccc1d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9a5c13560684ed288aed52bad065e9c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:18.281016 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6debf4fc-0188-40ee-b9f8-7ea128ccc1d5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:18.378316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb423ca-ba26-4003-8859-c69e1d93dbef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:18.379342 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb423ca-ba26-4003-8859-c69e1d93dbef service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:18.379767 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb423ca-ba26-4003-8859-c69e1d93dbef service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:18.380148 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb423ca-ba26-4003-8859-c69e1d93dbef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:18.383260 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-beb423ca-ba26-4003-8859-c69e1d93dbef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:18.422246 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6debf4fc-0188-40ee-b9f8-7ea128ccc1d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9a5c13560684ed288aed52bad065e9c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:18.438004 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6debf4fc-0188-40ee-b9f8-7ea128ccc1d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9a5c13560684ed288aed52bad065e9c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:18.463083 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6debf4fc-0188-40ee-b9f8-7ea128ccc1d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9a5c13560684ed288aed52bad065e9c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:18.464230 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6debf4fc-0188-40ee-b9f8-7ea128ccc1d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9a5c13560684ed288aed52bad065e9c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:18.465066 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6debf4fc-0188-40ee-b9f8-7ea128ccc1d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9a5c13560684ed288aed52bad065e9c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:18.502489 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:27:18.556255 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1095/2190] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:27:18 2022] DELETE /identity/v3/users/a9a5c13560684ed288aed52bad065e9c => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:27:18.560214 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77b303-2657-4ecb-b61b-60e9a76f454b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:18.560214 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77b303-2657-4ecb-b61b-60e9a76f454b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:18.560214 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77b303-2657-4ecb-b61b-60e9a76f454b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:18.600254 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:27:18.614333 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-cc77b303-2657-4ecb-b61b-60e9a76f454b None None] MFA Rules not processed for user `324738dd872e445ba845a101f45fa595`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:18.618242 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cc77b303-2657-4ecb-b61b-60e9a76f454b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:18.680010 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb423ca-ba26-4003-8859-c69e1d93dbef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:18.681808 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb423ca-ba26-4003-8859-c69e1d93dbef service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:18.718836 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1096/2191] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:27:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 916 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:27:18.721481 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb423ca-ba26-4003-8859-c69e1d93dbef service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:18.730945 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-beb423ca-ba26-4003-8859-c69e1d93dbef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:18.803730 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1096/2192] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:27:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:27:18.995448 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb423ca-ba26-4003-8859-c69e1d93dbef service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:19.375764 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c873dd-1668-4490-878b-af4f4f5120c3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0422d6695bf14925bf0f993cb16e64ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:19.377563 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c873dd-1668-4490-878b-af4f4f5120c3 service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:19.378222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c873dd-1668-4490-878b-af4f4f5120c3 service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:19.379073 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c873dd-1668-4490-878b-af4f4f5120c3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:19.405062 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-24c873dd-1668-4490-878b-af4f4f5120c3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:19.444358 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c873dd-1668-4490-878b-af4f4f5120c3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:19.453541 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c873dd-1668-4490-878b-af4f4f5120c3 service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:19.486262 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1097/2193] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:27:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 767 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:27:20.405120 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc77b303-2657-4ecb-b61b-60e9a76f454b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:20.408447 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cc77b303-2657-4ecb-b61b-60e9a76f454b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:20.641042 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc77b303-2657-4ecb-b61b-60e9a76f454b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:20.654885 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c80ef-932f-44de-9a1b-f9acb3beca14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:20.661931 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c80ef-932f-44de-9a1b-f9acb3beca14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:20.663089 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c80ef-932f-44de-9a1b-f9acb3beca14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:20.663917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c80ef-932f-44de-9a1b-f9acb3beca14 admin admin] PATH_INFO: `/v3/projects/c44d7520fb7e456dbeadc809570083b6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:20.666718 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105c80ef-932f-44de-9a1b-f9acb3beca14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c44d7520fb7e456dbeadc809570083b6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:20.669518 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105c80ef-932f-44de-9a1b-f9acb3beca14 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:20.778681 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-105c80ef-932f-44de-9a1b-f9acb3beca14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c44d7520fb7e456dbeadc809570083b6', 'request_id': 'req-105c80ef-932f-44de-9a1b-f9acb3beca14'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:20.790995 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-105c80ef-932f-44de-9a1b-f9acb3beca14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c44d7520fb7e456dbeadc809570083b6', 'request_id': 'req-105c80ef-932f-44de-9a1b-f9acb3beca14'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:20.800821 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-105c80ef-932f-44de-9a1b-f9acb3beca14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c44d7520fb7e456dbeadc809570083b6', 'request_id': 'req-105c80ef-932f-44de-9a1b-f9acb3beca14'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:20.824277 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-105c80ef-932f-44de-9a1b-f9acb3beca14 admin admin] The token cache is being invalidate because project c44d7520fb7e456dbeadc809570083b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:27:20.825056 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-105c80ef-932f-44de-9a1b-f9acb3beca14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:20.826442 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-105c80ef-932f-44de-9a1b-f9acb3beca14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:20.833769 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1097/2194] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:27:20 2022] DELETE /identity/v3/projects/c44d7520fb7e456dbeadc809570083b6 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:27:20.836637 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969fe5a-8712-45b0-a5fe-a28afeb13603 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:20.837879 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969fe5a-8712-45b0-a5fe-a28afeb13603 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:20.837879 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969fe5a-8712-45b0-a5fe-a28afeb13603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:20.889504 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7969fe5a-8712-45b0-a5fe-a28afeb13603 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:20.893330 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7969fe5a-8712-45b0-a5fe-a28afeb13603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:21.085807 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1098/2195] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:27:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:27:21.098968 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36adb15c-695e-43c6-8a8e-13f3db7d1735 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:21.099410 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36adb15c-695e-43c6-8a8e-13f3db7d1735 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:21.099627 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36adb15c-695e-43c6-8a8e-13f3db7d1735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:21.135503 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-36adb15c-695e-43c6-8a8e-13f3db7d1735 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:21.135937 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-36adb15c-695e-43c6-8a8e-13f3db7d1735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:21.160727 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1098/2196] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:27:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:27:21.169451 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969fe5a-8712-45b0-a5fe-a28afeb13603 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:21.191354 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969fe5a-8712-45b0-a5fe-a28afeb13603 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:21.201162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef39cc8-0137-421f-b7be-c119f60ec138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:21.203154 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef39cc8-0137-421f-b7be-c119f60ec138 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:21.203879 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef39cc8-0137-421f-b7be-c119f60ec138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:21.205363 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef39cc8-0137-421f-b7be-c119f60ec138 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:21.206869 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef39cc8-0137-421f-b7be-c119f60ec138 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:27:21.208324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef39cc8-0137-421f-b7be-c119f60ec138 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:21.212486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef39cc8-0137-421f-b7be-c119f60ec138 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:21.244404 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1099/2197] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:27:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:27:21.251417 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36adb15c-695e-43c6-8a8e-13f3db7d1735 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:21.279723 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36adb15c-695e-43c6-8a8e-13f3db7d1735 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:21.295218 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fbc742-2037-4df6-8dac-c61b3bdd00b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:21.295218 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fbc742-2037-4df6-8dac-c61b3bdd00b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:21.295218 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fbc742-2037-4df6-8dac-c61b3bdd00b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:21.295218 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fbc742-2037-4df6-8dac-c61b3bdd00b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:21.296698 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fbc742-2037-4df6-8dac-c61b3bdd00b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:21.298639 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fbc742-2037-4df6-8dac-c61b3bdd00b2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:21.330146 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:27:21.376659 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1099/2198] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:27:21 2022] POST /identity/v3/projects => generated 383 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:27:21.383059 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef39cc8-0137-421f-b7be-c119f60ec138 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:21.393280 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:27:21.404444 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef39cc8-0137-421f-b7be-c119f60ec138 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:21.622522 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc83132d-684a-4130-b4fe-7846212b1155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:21.634738 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83132d-684a-4130-b4fe-7846212b1155 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:21.635669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83132d-684a-4130-b4fe-7846212b1155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:21.644599 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83132d-684a-4130-b4fe-7846212b1155 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:21.648208 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc83132d-684a-4130-b4fe-7846212b1155 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:21.654539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc83132d-684a-4130-b4fe-7846212b1155 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:21.758272 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fbc742-2037-4df6-8dac-c61b3bdd00b2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:21.777074 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1100/2199] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:27:21 2022] POST /identity/v3/users => generated 292 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:27:21.807631 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fbc742-2037-4df6-8dac-c61b3bdd00b2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:21.837727 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c702e8f-361a-4bea-a823-c0186c84ff86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:21.841822 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c702e8f-361a-4bea-a823-c0186c84ff86 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:21.844270 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c702e8f-361a-4bea-a823-c0186c84ff86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:21.845201 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c702e8f-361a-4bea-a823-c0186c84ff86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:21.849248 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c702e8f-361a-4bea-a823-c0186c84ff86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:21.888934 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c702e8f-361a-4bea-a823-c0186c84ff86 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:21.929838 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1100/2200] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:27:21.935947 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc83132d-684a-4130-b4fe-7846212b1155 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:21.967180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc83132d-684a-4130-b4fe-7846212b1155 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:21.987815 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a13753-f9df-4c78-ac5d-e8ededc737bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:21.991581 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a13753-f9df-4c78-ac5d-e8ededc737bf admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:21.992373 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a13753-f9df-4c78-ac5d-e8ededc737bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:21.993002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a13753-f9df-4c78-ac5d-e8ededc737bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:21.994494 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a13753-f9df-4c78-ac5d-e8ededc737bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:21.999361 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a13753-f9df-4c78-ac5d-e8ededc737bf admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:22.078952 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1101/2201] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:27:22.082915 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c702e8f-361a-4bea-a823-c0186c84ff86 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:22.124302 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c702e8f-361a-4bea-a823-c0186c84ff86 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:22.163597 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d0d07a-c8bd-4345-b300-dfaaf7c0daf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:22.171376 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d0d07a-c8bd-4345-b300-dfaaf7c0daf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:22.175776 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d0d07a-c8bd-4345-b300-dfaaf7c0daf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:22.177762 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d0d07a-c8bd-4345-b300-dfaaf7c0daf3 admin admin] PATH_INFO: `/v3/projects/8b5003810fc84f68a11f3313f19b3902/users/58039eeaa31e4dcc9840b8c76f9feb63/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:22.236744 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d0d07a-c8bd-4345-b300-dfaaf7c0daf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b5003810fc84f68a11f3313f19b3902, user_id=58039eeaa31e4dcc9840b8c76f9feb63, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:22.242848 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d0d07a-c8bd-4345-b300-dfaaf7c0daf3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:22.309608 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1101/2202] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:27:22 2022] PUT /identity/v3/projects/8b5003810fc84f68a11f3313f19b3902/users/58039eeaa31e4dcc9840b8c76f9feb63/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:27:22.420810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306f476-3a2c-4924-844f-9461df8b7825 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:22.421923 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306f476-3a2c-4924-844f-9461df8b7825 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:22.422514 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306f476-3a2c-4924-844f-9461df8b7825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:22.508757 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7306f476-3a2c-4924-844f-9461df8b7825 None None] MFA Rules not processed for user `58039eeaa31e4dcc9840b8c76f9feb63`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:22.516724 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7306f476-3a2c-4924-844f-9461df8b7825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:22.821224 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1102/2203] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:27:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:27:22.846961 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d0d07a-c8bd-4345-b300-dfaaf7c0daf3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:22.853166 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a0d0d07a-c8bd-4345-b300-dfaaf7c0daf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:23.138630 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d0d07a-c8bd-4345-b300-dfaaf7c0daf3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:23.149745 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3093dec-249e-43be-8628-a2fe53167491 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:23.151765 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3093dec-249e-43be-8628-a2fe53167491 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:23.152931 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3093dec-249e-43be-8628-a2fe53167491 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:23.154105 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3093dec-249e-43be-8628-a2fe53167491 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:23.178619 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3093dec-249e-43be-8628-a2fe53167491 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:23.181973 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3093dec-249e-43be-8628-a2fe53167491 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:23.212025 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1102/2204] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:27:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:27:24.241718 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69245402-6556-4ceb-b2e3-3000e07365ba None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:24.242303 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69245402-6556-4ceb-b2e3-3000e07365ba None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:24.242539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69245402-6556-4ceb-b2e3-3000e07365ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:24.281962 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-69245402-6556-4ceb-b2e3-3000e07365ba None None] MFA Rules not processed for user `581ae6d40d34447da6651f3b2c21bdf0`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:24.284246 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-69245402-6556-4ceb-b2e3-3000e07365ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:24.422198 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1103/2205] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:27:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:27:26.825802 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3093dec-249e-43be-8628-a2fe53167491 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:26.968602 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3093dec-249e-43be-8628-a2fe53167491 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:26.988829 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f3a85-062d-4338-80d0-92347b9283b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:26.993249 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f3a85-062d-4338-80d0-92347b9283b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:26.994182 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f3a85-062d-4338-80d0-92347b9283b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:26.995415 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f3a85-062d-4338-80d0-92347b9283b8 admin admin] PATH_INFO: `/v3/users/58039eeaa31e4dcc9840b8c76f9feb63` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:26.999548 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2f3a85-062d-4338-80d0-92347b9283b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58039eeaa31e4dcc9840b8c76f9feb63)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:27.004291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2f3a85-062d-4338-80d0-92347b9283b8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:27.148097 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3d2f3a85-062d-4338-80d0-92347b9283b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58039eeaa31e4dcc9840b8c76f9feb63'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:27.179583 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3d2f3a85-062d-4338-80d0-92347b9283b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58039eeaa31e4dcc9840b8c76f9feb63'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:27.192052 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3d2f3a85-062d-4338-80d0-92347b9283b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58039eeaa31e4dcc9840b8c76f9feb63'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:27.193937 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3d2f3a85-062d-4338-80d0-92347b9283b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58039eeaa31e4dcc9840b8c76f9feb63'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:27.194810 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3d2f3a85-062d-4338-80d0-92347b9283b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58039eeaa31e4dcc9840b8c76f9feb63'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:27.199610 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:27:27.242377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1103/2206] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:27:26 2022] DELETE /identity/v3/users/58039eeaa31e4dcc9840b8c76f9feb63 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:27:27.249972 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf7440-e93a-4660-8a00-31650b3cd9de None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:27.251741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf7440-e93a-4660-8a00-31650b3cd9de None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:27.252491 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf7440-e93a-4660-8a00-31650b3cd9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:27.291728 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-fdcf7440-e93a-4660-8a00-31650b3cd9de None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:27.293142 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fdcf7440-e93a-4660-8a00-31650b3cd9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:27.308392 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:27:27.443731 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1104/2207] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:27:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:27:27.456454 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f3a85-062d-4338-80d0-92347b9283b8 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:27.459674 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3d2f3a85-062d-4338-80d0-92347b9283b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:27.612609 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f3a85-062d-4338-80d0-92347b9283b8 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:27.619987 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c53874-e500-4338-a411-b50ff629db2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:27.620851 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c53874-e500-4338-a411-b50ff629db2d service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:27.620990 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c53874-e500-4338-a411-b50ff629db2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:27.621230 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c53874-e500-4338-a411-b50ff629db2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:27.642790 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c53874-e500-4338-a411-b50ff629db2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:27.666335 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c53874-e500-4338-a411-b50ff629db2d service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:27.733311 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1104/2208] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:27:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:27:28.573340 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcf7440-e93a-4660-8a00-31650b3cd9de None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:28.576291 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fdcf7440-e93a-4660-8a00-31650b3cd9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:28.629748 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcf7440-e93a-4660-8a00-31650b3cd9de None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:28.640557 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f23690-f003-43a6-a032-60a46131397e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:28.644473 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f23690-f003-43a6-a032-60a46131397e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:28.645370 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f23690-f003-43a6-a032-60a46131397e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:28.646158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f23690-f003-43a6-a032-60a46131397e admin admin] PATH_INFO: `/v3/projects/8b5003810fc84f68a11f3313f19b3902` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:28.648965 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f23690-f003-43a6-a032-60a46131397e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b5003810fc84f68a11f3313f19b3902)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:28.654547 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f23690-f003-43a6-a032-60a46131397e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:28.801497 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c4f23690-f003-43a6-a032-60a46131397e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b5003810fc84f68a11f3313f19b3902', 'request_id': 'req-c4f23690-f003-43a6-a032-60a46131397e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:28.821924 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c4f23690-f003-43a6-a032-60a46131397e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b5003810fc84f68a11f3313f19b3902', 'request_id': 'req-c4f23690-f003-43a6-a032-60a46131397e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:28.838819 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c4f23690-f003-43a6-a032-60a46131397e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b5003810fc84f68a11f3313f19b3902', 'request_id': 'req-c4f23690-f003-43a6-a032-60a46131397e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:28.866226 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c4f23690-f003-43a6-a032-60a46131397e admin admin] The token cache is being invalidate because project 8b5003810fc84f68a11f3313f19b3902 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:27:28.866716 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c4f23690-f003-43a6-a032-60a46131397e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:28.868628 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c4f23690-f003-43a6-a032-60a46131397e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:28.872116 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1105/2209] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:27:28 2022] DELETE /identity/v3/projects/8b5003810fc84f68a11f3313f19b3902 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:27:37.689084 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=33,keystone:DELETE=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:27:38.862735 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:27:45.942752 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8575c-9f37-4ceb-b428-f6916386ab9d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:45.943893 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8575c-9f37-4ceb-b428-f6916386ab9d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:45.943893 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8575c-9f37-4ceb-b428-f6916386ab9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:45.980191 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-aca8575c-9f37-4ceb-b428-f6916386ab9d None None] MFA Rules not processed for user `6ecf34ceee294b53b5be7c1723b42950`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:45.982368 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-aca8575c-9f37-4ceb-b428-f6916386ab9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:46.119977 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1105/2210] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:27:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:27:46.704906 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280853ed-14f2-429d-b1a1-65b10031aec4 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:46.705810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280853ed-14f2-429d-b1a1-65b10031aec4 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:46.706582 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280853ed-14f2-429d-b1a1-65b10031aec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:46.750961 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-280853ed-14f2-429d-b1a1-65b10031aec4 None None] MFA Rules not processed for user `1c0772205ae24713b8901666a87018b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:46.753934 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-280853ed-14f2-429d-b1a1-65b10031aec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:46.868660 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1106/2211] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Jul 25 19:27:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:27:46.949734 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8575c-9f37-4ceb-b428-f6916386ab9d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:46.952409 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-aca8575c-9f37-4ceb-b428-f6916386ab9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:47.143888 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8575c-9f37-4ceb-b428-f6916386ab9d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:47.156948 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cde2c9-6ee6-45f8-9493-a5c7d3757e5d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feafcf1b5a9844fe9738823fddd07e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:47.158226 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cde2c9-6ee6-45f8-9493-a5c7d3757e5d service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:47.158531 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cde2c9-6ee6-45f8-9493-a5c7d3757e5d service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:47.158987 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cde2c9-6ee6-45f8-9493-a5c7d3757e5d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:47.203546 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cde2c9-6ee6-45f8-9493-a5c7d3757e5d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:47.206457 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cde2c9-6ee6-45f8-9493-a5c7d3757e5d service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:47.227540 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1106/2212] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Jul 25 19:27:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 19:27:48.543684 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6075fb-82f1-46da-898a-ad063a586c3f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:48.544996 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6075fb-82f1-46da-898a-ad063a586c3f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:48.545199 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6075fb-82f1-46da-898a-ad063a586c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:48.578387 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0d6075fb-82f1-46da-898a-ad063a586c3f None None] MFA Rules not processed for user `1c0772205ae24713b8901666a87018b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:48.580326 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0d6075fb-82f1-46da-898a-ad063a586c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:48.594778 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1107/2213] 10.222.0.18 () {58 vars in 967 bytes} [Mon Jul 25 19:27:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:27:48.620308 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cde2c9-6ee6-45f8-9493-a5c7d3757e5d service swift] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:48.651866 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cde2c9-6ee6-45f8-9493-a5c7d3757e5d service swift] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:48.663310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d9836-9f93-47e7-a5ef-c0fda36de849 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feafcf1b5a9844fe9738823fddd07e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:48.664867 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d9836-9f93-47e7-a5ef-c0fda36de849 service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:48.665580 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d9836-9f93-47e7-a5ef-c0fda36de849 service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:48.666300 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d9836-9f93-47e7-a5ef-c0fda36de849 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:48.692569 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d9836-9f93-47e7-a5ef-c0fda36de849 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:48.696323 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d9836-9f93-47e7-a5ef-c0fda36de849 service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:48.719031 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1107/2214] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Jul 25 19:27:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 19:27:49.388636 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660d33c-5e9e-49a7-955f-78e670d7ad3b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:49.389070 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660d33c-5e9e-49a7-955f-78e670d7ad3b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:49.389481 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660d33c-5e9e-49a7-955f-78e670d7ad3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:49.422602 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5660d33c-5e9e-49a7-955f-78e670d7ad3b None None] MFA Rules not processed for user `1c0772205ae24713b8901666a87018b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:49.424647 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5660d33c-5e9e-49a7-955f-78e670d7ad3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:49.439767 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1108/2215] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Jul 25 19:27:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:27:49.457478 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d9836-9f93-47e7-a5ef-c0fda36de849 service swift] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:49.477436 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d9836-9f93-47e7-a5ef-c0fda36de849 service swift] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:49.487600 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9368a91-7a6d-41fe-a8d2-bbb1364708fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feafcf1b5a9844fe9738823fddd07e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:49.488656 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9368a91-7a6d-41fe-a8d2-bbb1364708fa service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:49.489186 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9368a91-7a6d-41fe-a8d2-bbb1364708fa service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:49.490951 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9368a91-7a6d-41fe-a8d2-bbb1364708fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:49.509422 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9368a91-7a6d-41fe-a8d2-bbb1364708fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:49.513115 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9368a91-7a6d-41fe-a8d2-bbb1364708fa service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:49.533217 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1108/2216] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Jul 25 19:27:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 25 19:27:50.248237 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f51a6-00b5-4d83-98f5-6e2fda4d60e4 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:50.249066 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f51a6-00b5-4d83-98f5-6e2fda4d60e4 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:50.249248 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f51a6-00b5-4d83-98f5-6e2fda4d60e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:50.280070 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-488f51a6-00b5-4d83-98f5-6e2fda4d60e4 None None] MFA Rules not processed for user `1c0772205ae24713b8901666a87018b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:50.282445 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-488f51a6-00b5-4d83-98f5-6e2fda4d60e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:50.298672 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1109/2217] 10.222.0.18 () {58 vars in 967 bytes} [Mon Jul 25 19:27:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:27:50.318413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9368a91-7a6d-41fe-a8d2-bbb1364708fa service swift] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:50.343544 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9368a91-7a6d-41fe-a8d2-bbb1364708fa service swift] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:50.357487 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52669218-9626-4b68-8f2b-808982a9680e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feafcf1b5a9844fe9738823fddd07e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:50.359909 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52669218-9626-4b68-8f2b-808982a9680e service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:50.360274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52669218-9626-4b68-8f2b-808982a9680e service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:50.361208 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52669218-9626-4b68-8f2b-808982a9680e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:50.380355 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52669218-9626-4b68-8f2b-808982a9680e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:50.383977 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52669218-9626-4b68-8f2b-808982a9680e service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:50.397992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1109/2218] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Jul 25 19:27:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 19:27:53.505669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f51a6-00b5-4d83-98f5-6e2fda4d60e4 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:53.510832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-488f51a6-00b5-4d83-98f5-6e2fda4d60e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:53.668650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f51a6-00b5-4d83-98f5-6e2fda4d60e4 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:53.677597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e0b8e-9402-4889-9705-26f71272aa06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:53.679450 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e0b8e-9402-4889-9705-26f71272aa06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:53.679898 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e0b8e-9402-4889-9705-26f71272aa06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:53.680419 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e0b8e-9402-4889-9705-26f71272aa06 admin admin] PATH_INFO: `/v3/users/dc63fdeacd514eac8e4a5882190f8d66` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:53.683181 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28e0b8e-9402-4889-9705-26f71272aa06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc63fdeacd514eac8e4a5882190f8d66)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:53.687472 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28e0b8e-9402-4889-9705-26f71272aa06 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:53.806125 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f28e0b8e-9402-4889-9705-26f71272aa06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc63fdeacd514eac8e4a5882190f8d66'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:53.825424 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f28e0b8e-9402-4889-9705-26f71272aa06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc63fdeacd514eac8e4a5882190f8d66'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:53.849944 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f28e0b8e-9402-4889-9705-26f71272aa06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc63fdeacd514eac8e4a5882190f8d66'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:53.851333 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f28e0b8e-9402-4889-9705-26f71272aa06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc63fdeacd514eac8e4a5882190f8d66'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:53.884147 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f28e0b8e-9402-4889-9705-26f71272aa06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc63fdeacd514eac8e4a5882190f8d66'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:53.899762 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1110/2219] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:27:53 2022] DELETE /identity/v3/users/dc63fdeacd514eac8e4a5882190f8d66 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:27:53.908639 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d629073-2ebf-4e24-8153-f7142f7ce790 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:53.909444 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d629073-2ebf-4e24-8153-f7142f7ce790 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:53.909638 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d629073-2ebf-4e24-8153-f7142f7ce790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:53.951149 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9d629073-2ebf-4e24-8153-f7142f7ce790 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:53.953861 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9d629073-2ebf-4e24-8153-f7142f7ce790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:54.100322 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:27:54.142542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1110/2220] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:27:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:27:54.190589 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e0b8e-9402-4889-9705-26f71272aa06 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:54.195573 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f28e0b8e-9402-4889-9705-26f71272aa06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:54.212950 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:27:54.401937 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e0b8e-9402-4889-9705-26f71272aa06 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:54.420483 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3d92a-4b83-4d0b-b59d-7bde495d506c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:54.423373 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3d92a-4b83-4d0b-b59d-7bde495d506c service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:54.426594 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3d92a-4b83-4d0b-b59d-7bde495d506c service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:54.428397 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3d92a-4b83-4d0b-b59d-7bde495d506c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:54.475560 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e3d92a-4b83-4d0b-b59d-7bde495d506c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:54.478236 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e3d92a-4b83-4d0b-b59d-7bde495d506c service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:54.520665 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1111/2221] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:27:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:27:55.371447 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d629073-2ebf-4e24-8153-f7142f7ce790 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:55.374678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9d629073-2ebf-4e24-8153-f7142f7ce790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:55.419671 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d629073-2ebf-4e24-8153-f7142f7ce790 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:55.429486 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1757eb-ffe2-4628-8b35-ecb67d2b3cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:55.436495 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1757eb-ffe2-4628-8b35-ecb67d2b3cbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:55.438881 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1757eb-ffe2-4628-8b35-ecb67d2b3cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:55.440120 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1757eb-ffe2-4628-8b35-ecb67d2b3cbe admin admin] PATH_INFO: `/v3/projects/a092c7d1ac694eab812fe205ccbe5434` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:55.445436 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1757eb-ffe2-4628-8b35-ecb67d2b3cbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=a092c7d1ac694eab812fe205ccbe5434)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:55.450299 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1757eb-ffe2-4628-8b35-ecb67d2b3cbe admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:55.697282 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-de1757eb-ffe2-4628-8b35-ecb67d2b3cbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a092c7d1ac694eab812fe205ccbe5434', 'request_id': 'req-de1757eb-ffe2-4628-8b35-ecb67d2b3cbe'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:55.717998 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-de1757eb-ffe2-4628-8b35-ecb67d2b3cbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a092c7d1ac694eab812fe205ccbe5434', 'request_id': 'req-de1757eb-ffe2-4628-8b35-ecb67d2b3cbe'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:55.730387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-de1757eb-ffe2-4628-8b35-ecb67d2b3cbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a092c7d1ac694eab812fe205ccbe5434', 'request_id': 'req-de1757eb-ffe2-4628-8b35-ecb67d2b3cbe'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:55.757779 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-de1757eb-ffe2-4628-8b35-ecb67d2b3cbe admin admin] The token cache is being invalidate because project a092c7d1ac694eab812fe205ccbe5434 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:27:55.758065 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-de1757eb-ffe2-4628-8b35-ecb67d2b3cbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:55.758842 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-de1757eb-ffe2-4628-8b35-ecb67d2b3cbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:27:55.765595 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1111/2222] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:27:55 2022] DELETE /identity/v3/projects/a092c7d1ac694eab812fe205ccbe5434 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:27:55.769317 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6151819-294f-4017-9e49-a84b26e321d0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:55.770033 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6151819-294f-4017-9e49-a84b26e321d0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:55.770661 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6151819-294f-4017-9e49-a84b26e321d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:55.810214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d6151819-294f-4017-9e49-a84b26e321d0 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:55.811634 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d6151819-294f-4017-9e49-a84b26e321d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:55.948608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1112/2223] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:27:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:27:55.958493 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e20cc-159e-49a8-96ac-4514489e3e95 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:55.959862 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e20cc-159e-49a8-96ac-4514489e3e95 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:55.960506 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e20cc-159e-49a8-96ac-4514489e3e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:56.022181 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-042e20cc-159e-49a8-96ac-4514489e3e95 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:56.023632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-042e20cc-159e-49a8-96ac-4514489e3e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:56.048542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1112/2224] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:27:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:27:56.058164 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6151819-294f-4017-9e49-a84b26e321d0 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:56.078065 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6151819-294f-4017-9e49-a84b26e321d0 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:56.086722 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e16782-ffc5-4453-af1c-dedef933730a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:56.088010 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e16782-ffc5-4453-af1c-dedef933730a admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:56.088534 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e16782-ffc5-4453-af1c-dedef933730a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:56.089296 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e16782-ffc5-4453-af1c-dedef933730a admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:56.090747 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e16782-ffc5-4453-af1c-dedef933730a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:27:56.091244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e16782-ffc5-4453-af1c-dedef933730a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:56.094648 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e16782-ffc5-4453-af1c-dedef933730a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:56.132089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1113/2225] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:27:56.138108 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e20cc-159e-49a8-96ac-4514489e3e95 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:56.164711 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e20cc-159e-49a8-96ac-4514489e3e95 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:56.175710 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a49a3-16f1-45fd-9eea-80c815f5746a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:56.177026 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a49a3-16f1-45fd-9eea-80c815f5746a admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:56.177513 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a49a3-16f1-45fd-9eea-80c815f5746a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:56.177994 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a49a3-16f1-45fd-9eea-80c815f5746a admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:56.179326 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130a49a3-16f1-45fd-9eea-80c815f5746a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:56.183302 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130a49a3-16f1-45fd-9eea-80c815f5746a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:56.265521 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1113/2226] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:27:56 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:27:56.267912 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e16782-ffc5-4453-af1c-dedef933730a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:56.295424 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e16782-ffc5-4453-af1c-dedef933730a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:56.432858 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e88c6e3-1cf4-488e-9bb1-6f459c6f9b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:56.435401 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88c6e3-1cf4-488e-9bb1-6f459c6f9b32 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:56.436140 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88c6e3-1cf4-488e-9bb1-6f459c6f9b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:56.436963 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88c6e3-1cf4-488e-9bb1-6f459c6f9b32 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:56.438269 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e88c6e3-1cf4-488e-9bb1-6f459c6f9b32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:56.445327 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e88c6e3-1cf4-488e-9bb1-6f459c6f9b32 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:56.507491 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1114/2227] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:27:56 2022] POST /identity/v3/users => generated 287 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:27:56.511495 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a49a3-16f1-45fd-9eea-80c815f5746a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:56.538639 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a49a3-16f1-45fd-9eea-80c815f5746a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:56.549747 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8269a0-4960-476c-ae5f-5c3657bb5af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:56.550501 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8269a0-4960-476c-ae5f-5c3657bb5af5 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:56.551869 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8269a0-4960-476c-ae5f-5c3657bb5af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:56.551869 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8269a0-4960-476c-ae5f-5c3657bb5af5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:56.552642 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8269a0-4960-476c-ae5f-5c3657bb5af5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:56.554572 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8269a0-4960-476c-ae5f-5c3657bb5af5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:56.575546 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1114/2228] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:27:56.579719 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e88c6e3-1cf4-488e-9bb1-6f459c6f9b32 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:56.596421 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e88c6e3-1cf4-488e-9bb1-6f459c6f9b32 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:56.603064 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d5d79-d7fe-447f-8433-b1e91b97b706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:56.603882 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d5d79-d7fe-447f-8433-b1e91b97b706 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:56.604177 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d5d79-d7fe-447f-8433-b1e91b97b706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:56.604465 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d5d79-d7fe-447f-8433-b1e91b97b706 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:56.605304 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d5d79-d7fe-447f-8433-b1e91b97b706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:56.607088 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d5d79-d7fe-447f-8433-b1e91b97b706 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:56.625946 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1115/2229] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:27:56.631201 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8269a0-4960-476c-ae5f-5c3657bb5af5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:56.647770 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8269a0-4960-476c-ae5f-5c3657bb5af5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:56.666282 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8523b5-553a-48a0-a8a4-43981a982f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:56.667481 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8523b5-553a-48a0-a8a4-43981a982f27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:56.667815 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8523b5-553a-48a0-a8a4-43981a982f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:56.668381 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8523b5-553a-48a0-a8a4-43981a982f27 admin admin] PATH_INFO: `/v3/projects/9166cf61c5d944fd9f1e2174c6544faa/users/574d0113dce74ab199c0162abe26934b/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:56.715105 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8523b5-553a-48a0-a8a4-43981a982f27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9166cf61c5d944fd9f1e2174c6544faa, user_id=574d0113dce74ab199c0162abe26934b, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:56.721190 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8523b5-553a-48a0-a8a4-43981a982f27 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:56.758646 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1115/2230] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:27:56 2022] PUT /identity/v3/projects/9166cf61c5d944fd9f1e2174c6544faa/users/574d0113dce74ab199c0162abe26934b/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:27:56.768657 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b810287-e454-488c-88f9-b1781b987ccd None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:56.773983 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b810287-e454-488c-88f9-b1781b987ccd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:56.773983 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b810287-e454-488c-88f9-b1781b987ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:56.825209 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6b810287-e454-488c-88f9-b1781b987ccd None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:27:56.826636 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6b810287-e454-488c-88f9-b1781b987ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:56.843798 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:27:56.992264 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1116/2231] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:27:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:27:57.004376 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8523b5-553a-48a0-a8a4-43981a982f27 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:27:57.010025 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1d8523b5-553a-48a0-a8a4-43981a982f27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:27:57.244830 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8523b5-553a-48a0-a8a4-43981a982f27 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:27:57.269236 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6be9d3-49c4-467e-9305-f3b070cb454f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:27:57.271627 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6be9d3-49c4-467e-9305-f3b070cb454f service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:27:57.272145 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6be9d3-49c4-467e-9305-f3b070cb454f service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:27:57.272963 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6be9d3-49c4-467e-9305-f3b070cb454f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:27:57.297547 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6be9d3-49c4-467e-9305-f3b070cb454f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:27:57.302239 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6be9d3-49c4-467e-9305-f3b070cb454f service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:27:57.336326 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1116/2232] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:27:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:28:00.000057 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dffe39-158e-4cd8-9126-c320ae73631c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:00.001860 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dffe39-158e-4cd8-9126-c320ae73631c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:00.002635 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dffe39-158e-4cd8-9126-c320ae73631c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:00.044756 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-74dffe39-158e-4cd8-9126-c320ae73631c None None] MFA Rules not processed for user `1c0772205ae24713b8901666a87018b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:28:00.047501 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-74dffe39-158e-4cd8-9126-c320ae73631c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:00.236354 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1117/2233] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Jul 25 19:27:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:28:00.270700 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6be9d3-49c4-467e-9305-f3b070cb454f service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:00.273736 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-aa6be9d3-49c4-467e-9305-f3b070cb454f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:00.344433 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:28:00.495088 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6be9d3-49c4-467e-9305-f3b070cb454f service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:00.507177 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24958e93-90f6-476a-a700-53c9da80f298 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feafcf1b5a9844fe9738823fddd07e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:00.508741 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24958e93-90f6-476a-a700-53c9da80f298 service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:00.509255 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24958e93-90f6-476a-a700-53c9da80f298 service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:00.509759 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24958e93-90f6-476a-a700-53c9da80f298 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:00.544213 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24958e93-90f6-476a-a700-53c9da80f298 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:00.548502 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24958e93-90f6-476a-a700-53c9da80f298 service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:00.576052 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1117/2234] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Jul 25 19:28:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 19:28:09.773657 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7b29a-76a7-4c40-813c-f668dd0e8d21 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:09.775703 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7b29a-76a7-4c40-813c-f668dd0e8d21 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:09.776682 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7b29a-76a7-4c40-813c-f668dd0e8d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:09.860907 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-dce7b29a-76a7-4c40-813c-f668dd0e8d21 None None] MFA Rules not processed for user `574d0113dce74ab199c0162abe26934b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:28:09.864279 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-dce7b29a-76a7-4c40-813c-f668dd0e8d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:10.067608 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24958e93-90f6-476a-a700-53c9da80f298 service swift] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:10.077204 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1118/2235] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:28:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:28:10.103939 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24958e93-90f6-476a-a700-53c9da80f298 service swift] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:10.123928 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7331b15-6d74-49c7-b740-38559540bc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:10.125156 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7331b15-6d74-49c7-b740-38559540bc1a admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:10.125686 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7331b15-6d74-49c7-b740-38559540bc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:10.126356 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7331b15-6d74-49c7-b740-38559540bc1a admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:10.128120 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7331b15-6d74-49c7-b740-38559540bc1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:10.137886 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7331b15-6d74-49c7-b740-38559540bc1a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:10.232068 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1118/2236] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:28:10 2022] POST /identity/v3/projects => generated 371 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:28:10.234437 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce7b29a-76a7-4c40-813c-f668dd0e8d21 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:10.282504 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce7b29a-76a7-4c40-813c-f668dd0e8d21 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:10.532895 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:28:10.610897 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803d8d3-2990-4b9b-9b40-3a5c4e9e00bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:10.616309 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803d8d3-2990-4b9b-9b40-3a5c4e9e00bb admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:10.616923 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803d8d3-2990-4b9b-9b40-3a5c4e9e00bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:10.617508 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803d8d3-2990-4b9b-9b40-3a5c4e9e00bb admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:10.623648 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d803d8d3-2990-4b9b-9b40-3a5c4e9e00bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:10.630552 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d803d8d3-2990-4b9b-9b40-3a5c4e9e00bb admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:10.708234 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1119/2237] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:28:10 2022] POST /identity/v3/users => generated 292 bytes in 474 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:28:10.711603 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7331b15-6d74-49c7-b740-38559540bc1a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:10.740335 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7331b15-6d74-49c7-b740-38559540bc1a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:10.750300 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4653dd4d-2306-46c5-8f06-bb2c91f84558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:10.752181 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653dd4d-2306-46c5-8f06-bb2c91f84558 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:10.752985 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653dd4d-2306-46c5-8f06-bb2c91f84558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:10.754837 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653dd4d-2306-46c5-8f06-bb2c91f84558 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:10.756939 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4653dd4d-2306-46c5-8f06-bb2c91f84558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:10.761009 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4653dd4d-2306-46c5-8f06-bb2c91f84558 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:10.864312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1119/2238] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:28:10.870468 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803d8d3-2990-4b9b-9b40-3a5c4e9e00bb admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:10.888601 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803d8d3-2990-4b9b-9b40-3a5c4e9e00bb admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:10.901130 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b24cdbe-91cd-4156-8e45-fb7773ab77ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:10.902061 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24cdbe-91cd-4156-8e45-fb7773ab77ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:10.902463 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24cdbe-91cd-4156-8e45-fb7773ab77ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:10.902637 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24cdbe-91cd-4156-8e45-fb7773ab77ae admin admin] PATH_INFO: `/v3/projects/e70d838c7e8b416791220cf1643b545e/users/064f1f93ebd44f4e89c3fb52322e42bb/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:10.945227 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b24cdbe-91cd-4156-8e45-fb7773ab77ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=e70d838c7e8b416791220cf1643b545e, user_id=064f1f93ebd44f4e89c3fb52322e42bb, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:10.948719 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b24cdbe-91cd-4156-8e45-fb7773ab77ae admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:11.009008 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1120/2239] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:28:10 2022] PUT /identity/v3/projects/e70d838c7e8b416791220cf1643b545e/users/064f1f93ebd44f4e89c3fb52322e42bb/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:28:11.012079 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4653dd4d-2306-46c5-8f06-bb2c91f84558 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:11.053316 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4653dd4d-2306-46c5-8f06-bb2c91f84558 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:11.360422 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13973748-cc56-43c6-a708-2c22fd71f7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:11.361550 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13973748-cc56-43c6-a708-2c22fd71f7d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:11.362037 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13973748-cc56-43c6-a708-2c22fd71f7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:11.362577 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13973748-cc56-43c6-a708-2c22fd71f7d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:11.364069 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13973748-cc56-43c6-a708-2c22fd71f7d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:11.368338 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13973748-cc56-43c6-a708-2c22fd71f7d2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:11.436763 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b24cdbe-91cd-4156-8e45-fb7773ab77ae admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:11.445099 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1120/2240] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:28:11.477991 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b24cdbe-91cd-4156-8e45-fb7773ab77ae admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:11.505602 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979fa68f-0a60-40ef-bdcf-72d92d92f1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:11.506740 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fa68f-0a60-40ef-bdcf-72d92d92f1a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:11.507036 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fa68f-0a60-40ef-bdcf-72d92d92f1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:11.508792 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fa68f-0a60-40ef-bdcf-72d92d92f1a8 admin admin] PATH_INFO: `/v3/projects/e70d838c7e8b416791220cf1643b545e/users/064f1f93ebd44f4e89c3fb52322e42bb/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:11.524682 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979fa68f-0a60-40ef-bdcf-72d92d92f1a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e70d838c7e8b416791220cf1643b545e, user_id=064f1f93ebd44f4e89c3fb52322e42bb, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:11.524682 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979fa68f-0a60-40ef-bdcf-72d92d92f1a8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:11.565644 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1121/2241] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:28:11 2022] PUT /identity/v3/projects/e70d838c7e8b416791220cf1643b545e/users/064f1f93ebd44f4e89c3fb52322e42bb/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:28:17.527415 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a25f4-60ce-4e58-aebf-8e4ea059efd7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:17.530140 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a25f4-60ce-4e58-aebf-8e4ea059efd7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:17.530836 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a25f4-60ce-4e58-aebf-8e4ea059efd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:17.620305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-321a25f4-60ce-4e58-aebf-8e4ea059efd7 None None] MFA Rules not processed for user `0ed3ed6183d34c73ba40d13af037f838`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:28:17.624870 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-321a25f4-60ce-4e58-aebf-8e4ea059efd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:17.763737 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1121/2242] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:28:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:28:21.542089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=22,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:28:27.745788 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:28:31.479665 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bca7cf-ea07-414b-a58f-20c62c2c4e4e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:31.480431 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bca7cf-ea07-414b-a58f-20c62c2c4e4e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:31.480626 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bca7cf-ea07-414b-a58f-20c62c2c4e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:31.535288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-13bca7cf-ea07-414b-a58f-20c62c2c4e4e None None] MFA Rules not processed for user `064f1f93ebd44f4e89c3fb52322e42bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:28:31.538129 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-13bca7cf-ea07-414b-a58f-20c62c2c4e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:31.687697 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1122/2243] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:28:31 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:28:31.704929 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a25f4-60ce-4e58-aebf-8e4ea059efd7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:31.707932 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-321a25f4-60ce-4e58-aebf-8e4ea059efd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:31.874554 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a25f4-60ce-4e58-aebf-8e4ea059efd7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:31.882641 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf81db-b6a5-416c-89bc-1ab09ef40c66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:31.883354 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf81db-b6a5-416c-89bc-1ab09ef40c66 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:31.883696 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf81db-b6a5-416c-89bc-1ab09ef40c66 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:31.883981 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf81db-b6a5-416c-89bc-1ab09ef40c66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:31.974706 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbf81db-b6a5-416c-89bc-1ab09ef40c66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:31.977495 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbf81db-b6a5-416c-89bc-1ab09ef40c66 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:31.997394 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1122/2244] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:28:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:28:34.992509 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8394d71-b495-4e5b-8e0c-41c8daa086f3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:34.993362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8394d71-b495-4e5b-8e0c-41c8daa086f3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:34.993362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8394d71-b495-4e5b-8e0c-41c8daa086f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:35.019107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b8394d71-b495-4e5b-8e0c-41c8daa086f3 None None] MFA Rules not processed for user `574d0113dce74ab199c0162abe26934b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:28:35.020475 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b8394d71-b495-4e5b-8e0c-41c8daa086f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:35.029007 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1123/2245] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:28:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:28:41.988333 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=56 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:28:45.013090 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=37 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:28:46.252709 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072a33a-0d10-4141-876a-22ce48b4c4ba None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:46.254711 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072a33a-0d10-4141-876a-22ce48b4c4ba None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:46.255186 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072a33a-0d10-4141-876a-22ce48b4c4ba None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:46.264214 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1123/2246] 10.222.0.18 () {54 vars in 921 bytes} [Mon Jul 25 19:28:46 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:28:46.293376 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1ffbf-245a-409b-ac66-9b716f58e2a7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:46.294133 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1ffbf-245a-409b-ac66-9b716f58e2a7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:46.294406 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1ffbf-245a-409b-ac66-9b716f58e2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:46.337857 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-74b1ffbf-245a-409b-ac66-9b716f58e2a7 None None] MFA Rules not processed for user `860e7d26ad364795bb25836aadd4ac7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:28:46.337857 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-74b1ffbf-245a-409b-ac66-9b716f58e2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:46.364877 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1124/2247] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Jul 25 19:28:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:28:46.387862 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072a33a-0d10-4141-876a-22ce48b4c4ba None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:46.391380 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f072a33a-0d10-4141-876a-22ce48b4c4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:46.523588 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072a33a-0d10-4141-876a-22ce48b4c4ba None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:46.526357 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffadb7dd-a084-4211-b926-0b83d92649a1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98560723b49940d38a6a5e9cf3f5d3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:46.527245 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffadb7dd-a084-4211-b926-0b83d92649a1 service glance] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:46.527668 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffadb7dd-a084-4211-b926-0b83d92649a1 service glance] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:46.528091 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffadb7dd-a084-4211-b926-0b83d92649a1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:46.549311 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffadb7dd-a084-4211-b926-0b83d92649a1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:46.552134 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffadb7dd-a084-4211-b926-0b83d92649a1 service glance] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:46.571545 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1124/2248] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:28:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:28:46.718431 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4946bf-4283-45c5-b18b-c70602380bbd None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:46.718925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4946bf-4283-45c5-b18b-c70602380bbd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:46.719898 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4946bf-4283-45c5-b18b-c70602380bbd None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:46.722073 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1125/2249] 10.222.0.18 () {54 vars in 921 bytes} [Mon Jul 25 19:28:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:28:46.730778 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d7378-2c17-4ab1-8645-05b55c06dfe7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:46.731965 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d7378-2c17-4ab1-8645-05b55c06dfe7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:46.732796 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d7378-2c17-4ab1-8645-05b55c06dfe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:46.781400 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-527d7378-2c17-4ab1-8645-05b55c06dfe7 None None] MFA Rules not processed for user `181c84715c094100b414cea3c4919eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:28:46.783650 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-527d7378-2c17-4ab1-8645-05b55c06dfe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:46.891873 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1125/2250] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Jul 25 19:28:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:28:46.910713 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4946bf-4283-45c5-b18b-c70602380bbd None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:46.929869 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4946bf-4283-45c5-b18b-c70602380bbd None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:46.936302 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fd753c-abf4-4a73-81ea-feeb30344f90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:46.937447 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd753c-abf4-4a73-81ea-feeb30344f90 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:46.937958 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd753c-abf4-4a73-81ea-feeb30344f90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:46.938465 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd753c-abf4-4a73-81ea-feeb30344f90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:46.956057 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fd753c-abf4-4a73-81ea-feeb30344f90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:46.958824 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fd753c-abf4-4a73-81ea-feeb30344f90 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:46.976255 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1126/2251] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:28:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:28:47.167733 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02207491-9b40-4e72-86ae-6bc3e2fe87a6 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:47.168028 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02207491-9b40-4e72-86ae-6bc3e2fe87a6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:47.168316 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02207491-9b40-4e72-86ae-6bc3e2fe87a6 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:47.169848 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1126/2252] 10.222.0.18 () {54 vars in 921 bytes} [Mon Jul 25 19:28:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:28:47.187831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17115a8e-e474-4989-8803-b321fcdc707f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:47.188470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17115a8e-e474-4989-8803-b321fcdc707f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:47.188885 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17115a8e-e474-4989-8803-b321fcdc707f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:47.217454 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-17115a8e-e474-4989-8803-b321fcdc707f None None] MFA Rules not processed for user `860e7d26ad364795bb25836aadd4ac7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:28:47.218721 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-17115a8e-e474-4989-8803-b321fcdc707f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:28:47.231886 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1127/2253] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Jul 25 19:28:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:28:47.247930 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02207491-9b40-4e72-86ae-6bc3e2fe87a6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:28:47.270553 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02207491-9b40-4e72-86ae-6bc3e2fe87a6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:28:47.277307 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40716b5d-2f9b-4127-9dc2-2afafdd39910 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:28:47.278447 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40716b5d-2f9b-4127-9dc2-2afafdd39910 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:28:47.278965 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40716b5d-2f9b-4127-9dc2-2afafdd39910 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:28:47.279477 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40716b5d-2f9b-4127-9dc2-2afafdd39910 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:28:47.304295 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40716b5d-2f9b-4127-9dc2-2afafdd39910 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:28:47.307722 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40716b5d-2f9b-4127-9dc2-2afafdd39910 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:28:47.334797 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1127/2254] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:28:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:28:57.208617 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=12 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:28:57.320223 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=55 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:29:08.008554 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5fe27-4ff6-4552-8877-76553bd50baa None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:08.009187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5fe27-4ff6-4552-8877-76553bd50baa None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:08.009187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5fe27-4ff6-4552-8877-76553bd50baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:08.036698 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-56a5fe27-4ff6-4552-8877-76553bd50baa None None] MFA Rules not processed for user `1c0772205ae24713b8901666a87018b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:29:08.038159 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-56a5fe27-4ff6-4552-8877-76553bd50baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:29:08.154339 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1128/2255] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Jul 25 19:29:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:29:08.237660 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40716b5d-2f9b-4127-9dc2-2afafdd39910 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:08.363632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40716b5d-2f9b-4127-9dc2-2afafdd39910 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:08.374647 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb52d3-9435-4294-bba2-66c4ebacaecb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feafcf1b5a9844fe9738823fddd07e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:08.376060 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb52d3-9435-4294-bba2-66c4ebacaecb service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:08.376667 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb52d3-9435-4294-bba2-66c4ebacaecb service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:08.377315 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb52d3-9435-4294-bba2-66c4ebacaecb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:08.396228 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb52d3-9435-4294-bba2-66c4ebacaecb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:08.399664 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb52d3-9435-4294-bba2-66c4ebacaecb service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:08.417251 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1128/2256] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Jul 25 19:29:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 25 19:29:08.687908 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b46c943-244d-4d42-a2be-0064d856d7a2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:08.688498 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b46c943-244d-4d42-a2be-0064d856d7a2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:08.688859 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b46c943-244d-4d42-a2be-0064d856d7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:08.746542 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2b46c943-244d-4d42-a2be-0064d856d7a2 None None] MFA Rules not processed for user `324738dd872e445ba845a101f45fa595`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:29:08.746542 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2b46c943-244d-4d42-a2be-0064d856d7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:29:08.878290 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1129/2257] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:29:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:29:18.412754 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=32 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:29:18.865886 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=50 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:29:31.920769 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb52d3-9435-4294-bba2-66c4ebacaecb service swift] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:31.923140 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-aefb52d3-9435-4294-bba2-66c4ebacaecb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:29:32.030983 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb52d3-9435-4294-bba2-66c4ebacaecb service swift] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:32.037945 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f0a21-fe09-4199-bfb3-d00c7e6eaa20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:32.039113 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f0a21-fe09-4199-bfb3-d00c7e6eaa20 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:32.039685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f0a21-fe09-4199-bfb3-d00c7e6eaa20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:32.040196 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f0a21-fe09-4199-bfb3-d00c7e6eaa20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:32.043829 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a95f0a21-fe09-4199-bfb3-d00c7e6eaa20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:29:32.071137 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95f0a21-fe09-4199-bfb3-d00c7e6eaa20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:32.074026 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95f0a21-fe09-4199-bfb3-d00c7e6eaa20 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:32.101223 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1129/2258] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:29:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:29:34.574935 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b46c943-244d-4d42-a2be-0064d856d7a2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:34.578352 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2b46c943-244d-4d42-a2be-0064d856d7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:29:34.604964 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b46c943-244d-4d42-a2be-0064d856d7a2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:34.616463 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5c29c-8111-42c6-a0f0-64e0043d5a0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:34.618305 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5c29c-8111-42c6-a0f0-64e0043d5a0b service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:34.619119 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5c29c-8111-42c6-a0f0-64e0043d5a0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:34.620140 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5c29c-8111-42c6-a0f0-64e0043d5a0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:34.623718 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-dca5c29c-8111-42c6-a0f0-64e0043d5a0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:29:34.670091 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5c29c-8111-42c6-a0f0-64e0043d5a0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:34.672328 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5c29c-8111-42c6-a0f0-64e0043d5a0b service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:34.691775 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1130/2259] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:29:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:29:42.087647 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=26 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:29:42.198569 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b9981-5fd4-46a5-b623-71a0237c768f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:42.199183 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b9981-5fd4-46a5-b623-71a0237c768f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:42.199703 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b9981-5fd4-46a5-b623-71a0237c768f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:42.237943 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1d5b9981-5fd4-46a5-b623-71a0237c768f None None] MFA Rules not processed for user `1c0772205ae24713b8901666a87018b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:29:42.240047 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1d5b9981-5fd4-46a5-b623-71a0237c768f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:29:42.250522 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1130/2260] 10.222.0.18 () {58 vars in 967 bytes} [Mon Jul 25 19:29:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:29:42.269029 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5c29c-8111-42c6-a0f0-64e0043d5a0b service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:29:42.285171 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5c29c-8111-42c6-a0f0-64e0043d5a0b service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:29:42.294196 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d6ed2-57b9-4b6b-8ecf-2c9847adcfa6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feafcf1b5a9844fe9738823fddd07e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:29:42.295242 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d6ed2-57b9-4b6b-8ecf-2c9847adcfa6 service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:29:42.295745 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d6ed2-57b9-4b6b-8ecf-2c9847adcfa6 service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:29:42.296192 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d6ed2-57b9-4b6b-8ecf-2c9847adcfa6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:29:42.311356 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d6ed2-57b9-4b6b-8ecf-2c9847adcfa6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:29:42.313500 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d6ed2-57b9-4b6b-8ecf-2c9847adcfa6 service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:29:42.330302 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1131/2261] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Jul 25 19:29:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 25 19:29:52.231111 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:29:52.323910 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=12 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:30:02.162152 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5b9981-5fd4-46a5-b623-71a0237c768f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:30:02.181229 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5b9981-5fd4-46a5-b623-71a0237c768f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:30:02.187692 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170cb6a0-119a-4e60-8675-eb75c057d4de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:30:02.188722 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170cb6a0-119a-4e60-8675-eb75c057d4de service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:30:02.189189 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170cb6a0-119a-4e60-8675-eb75c057d4de service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:30:02.189642 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170cb6a0-119a-4e60-8675-eb75c057d4de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:30:02.210479 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170cb6a0-119a-4e60-8675-eb75c057d4de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:30:02.213196 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170cb6a0-119a-4e60-8675-eb75c057d4de service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:30:02.234461 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1131/2262] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:30:02 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:30:02.827272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d6ed2-57b9-4b6b-8ecf-2c9847adcfa6 service swift] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:30:02.828973 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d85d6ed2-57b9-4b6b-8ecf-2c9847adcfa6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:30:02.850029 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d6ed2-57b9-4b6b-8ecf-2c9847adcfa6 service swift] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:30:02.857204 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6efb623-d277-4104-8cdb-1c9db016f713 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:30:02.858018 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efb623-d277-4104-8cdb-1c9db016f713 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:30:02.858518 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efb623-d277-4104-8cdb-1c9db016f713 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:30:02.858870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efb623-d277-4104-8cdb-1c9db016f713 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:30:02.861514 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d6efb623-d277-4104-8cdb-1c9db016f713 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:30:02.966626 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6efb623-d277-4104-8cdb-1c9db016f713 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:30:02.969336 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6efb623-d277-4104-8cdb-1c9db016f713 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:30:02.985708 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1132/2263] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:30:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:30:12.224721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:30:12.977582 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=28 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:30:23.109551 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170cb6a0-119a-4e60-8675-eb75c057d4de service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:30:23.178967 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170cb6a0-119a-4e60-8675-eb75c057d4de service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:30:23.183892 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f48f6ca-2214-4a7a-b776-f36b0786e406 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:30:23.184703 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48f6ca-2214-4a7a-b776-f36b0786e406 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:30:23.184998 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48f6ca-2214-4a7a-b776-f36b0786e406 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:30:23.185369 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48f6ca-2214-4a7a-b776-f36b0786e406 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:30:23.187362 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8f48f6ca-2214-4a7a-b776-f36b0786e406 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:30:23.232733 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f48f6ca-2214-4a7a-b776-f36b0786e406 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:30:23.234447 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f48f6ca-2214-4a7a-b776-f36b0786e406 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:30:23.246455 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1132/2264] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:30:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:30:28.006092 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6efb623-d277-4104-8cdb-1c9db016f713 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:30:28.086205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6efb623-d277-4104-8cdb-1c9db016f713 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:30:28.095139 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4edf8-3b58-4dab-9b0a-95990a75a114 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:30:28.096226 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4edf8-3b58-4dab-9b0a-95990a75a114 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:30:28.096674 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4edf8-3b58-4dab-9b0a-95990a75a114 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:30:28.097111 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4edf8-3b58-4dab-9b0a-95990a75a114 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:30:28.101076 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-caf4edf8-3b58-4dab-9b0a-95990a75a114 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:30:28.119846 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf4edf8-3b58-4dab-9b0a-95990a75a114 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:30:28.122964 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf4edf8-3b58-4dab-9b0a-95990a75a114 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:30:28.141532 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1133/2265] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:30:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:30:33.241325 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=27 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:30:38.132656 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=20 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:30:58.149252 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f48f6ca-2214-4a7a-b776-f36b0786e406 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:30:58.174654 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f48f6ca-2214-4a7a-b776-f36b0786e406 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:30:58.187157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e73bf3e-ad0f-46df-9816-bc8981e41e88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:30:58.188728 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73bf3e-ad0f-46df-9816-bc8981e41e88 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:30:58.189177 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73bf3e-ad0f-46df-9816-bc8981e41e88 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:30:58.189865 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73bf3e-ad0f-46df-9816-bc8981e41e88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:30:58.194729 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6e73bf3e-ad0f-46df-9816-bc8981e41e88 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:30:58.299067 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e73bf3e-ad0f-46df-9816-bc8981e41e88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:30:58.300755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e73bf3e-ad0f-46df-9816-bc8981e41e88 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:30:58.315463 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1133/2266] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:30:58 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:31:08.308800 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=28 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:32:17.057715 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4edf8-3b58-4dab-9b0a-95990a75a114 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:32:17.059544 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-caf4edf8-3b58-4dab-9b0a-95990a75a114 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:32:17.126633 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4edf8-3b58-4dab-9b0a-95990a75a114 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:32:17.132580 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd2db4-ee1a-4eb5-bc3a-c98e4a48b7c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:32:17.133722 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd2db4-ee1a-4eb5-bc3a-c98e4a48b7c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:32:17.134216 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd2db4-ee1a-4eb5-bc3a-c98e4a48b7c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:32:17.134696 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd2db4-ee1a-4eb5-bc3a-c98e4a48b7c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:32:17.138287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7cbd2db4-ee1a-4eb5-bc3a-c98e4a48b7c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:32:17.154923 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbd2db4-ee1a-4eb5-bc3a-c98e4a48b7c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:32:17.157631 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbd2db4-ee1a-4eb5-bc3a-c98e4a48b7c3 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:32:17.172597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1134/2267] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:32:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:32:19.391674 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e73bf3e-ad0f-46df-9816-bc8981e41e88 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:32:19.404808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e73bf3e-ad0f-46df-9816-bc8981e41e88 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:32:19.409554 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a55a2f2-44a3-4327-b1ae-f4edd4c4cb1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:32:19.410359 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55a2f2-44a3-4327-b1ae-f4edd4c4cb1a service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:32:19.410765 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55a2f2-44a3-4327-b1ae-f4edd4c4cb1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:32:19.411137 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55a2f2-44a3-4327-b1ae-f4edd4c4cb1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:32:19.413437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8a55a2f2-44a3-4327-b1ae-f4edd4c4cb1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:32:19.522838 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a55a2f2-44a3-4327-b1ae-f4edd4c4cb1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:32:19.525874 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a55a2f2-44a3-4327-b1ae-f4edd4c4cb1a service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:32:19.545434 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1134/2268] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:32:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:32:27.165682 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=15 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:32:29.536520 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=28 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:32:53.232217 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd2db4-ee1a-4eb5-bc3a-c98e4a48b7c3 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:32:53.284882 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd2db4-ee1a-4eb5-bc3a-c98e4a48b7c3 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:32:53.295012 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f160502a-b915-482b-bffb-6517be168d14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:32:53.296248 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160502a-b915-482b-bffb-6517be168d14 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:32:53.296772 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160502a-b915-482b-bffb-6517be168d14 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:32:53.297269 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160502a-b915-482b-bffb-6517be168d14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:32:53.300769 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f160502a-b915-482b-bffb-6517be168d14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:32:53.344930 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f160502a-b915-482b-bffb-6517be168d14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:32:53.348133 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f160502a-b915-482b-bffb-6517be168d14 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:32:53.368721 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1135/2269] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:32:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:33:03.359571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=18 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:33:19.409988 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a55a2f2-44a3-4327-b1ae-f4edd4c4cb1a service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:33:19.411603 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8a55a2f2-44a3-4327-b1ae-f4edd4c4cb1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:33:19.572340 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a55a2f2-44a3-4327-b1ae-f4edd4c4cb1a service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:33:19.578324 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f6e84-5302-4e0b-9c89-1b15d31f57e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0422d6695bf14925bf0f993cb16e64ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:33:19.579012 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f6e84-5302-4e0b-9c89-1b15d31f57e6 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:33:19.579301 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f6e84-5302-4e0b-9c89-1b15d31f57e6 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:33:19.579621 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f6e84-5302-4e0b-9c89-1b15d31f57e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:33:19.581646 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-178f6e84-5302-4e0b-9c89-1b15d31f57e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:33:19.600311 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f6e84-5302-4e0b-9c89-1b15d31f57e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:33:19.602322 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f6e84-5302-4e0b-9c89-1b15d31f57e6 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:33:19.618403 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1135/2270] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:33:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:33:29.618541 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=41 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:33:31.945496 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f160502a-b915-482b-bffb-6517be168d14 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:33:31.967145 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f160502a-b915-482b-bffb-6517be168d14 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:33:31.978414 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902843c5-d7dd-4ddb-96e4-326b07c4a2f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:33:31.979727 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902843c5-d7dd-4ddb-96e4-326b07c4a2f1 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:33:31.980253 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902843c5-d7dd-4ddb-96e4-326b07c4a2f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:33:31.980773 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902843c5-d7dd-4ddb-96e4-326b07c4a2f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:33:31.998563 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902843c5-d7dd-4ddb-96e4-326b07c4a2f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:33:32.001797 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902843c5-d7dd-4ddb-96e4-326b07c4a2f1 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:33:32.021605 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1136/2271] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:33:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:33:42.012101 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:34:33.807763 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f6e84-5302-4e0b-9c89-1b15d31f57e6 service placement] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:34:33.818781 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f6e84-5302-4e0b-9c89-1b15d31f57e6 service placement] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:34:33.823002 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d1b986-3b77-49e1-bf9d-b8f80f933334 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:34:33.823717 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1b986-3b77-49e1-bf9d-b8f80f933334 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:34:33.824042 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1b986-3b77-49e1-bf9d-b8f80f933334 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:34:33.824323 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1b986-3b77-49e1-bf9d-b8f80f933334 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:34:33.837085 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d1b986-3b77-49e1-bf9d-b8f80f933334 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:34:33.838559 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d1b986-3b77-49e1-bf9d-b8f80f933334 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:34:33.853813 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1136/2272] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:34:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:34:35.090843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902843c5-d7dd-4ddb-96e4-326b07c4a2f1 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:34:35.108155 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902843c5-d7dd-4ddb-96e4-326b07c4a2f1 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:34:35.115189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00581a-8cbb-4f6c-9d4c-be9357cd21ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:34:35.116647 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00581a-8cbb-4f6c-9d4c-be9357cd21ba service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:34:35.117344 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00581a-8cbb-4f6c-9d4c-be9357cd21ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:34:35.117845 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00581a-8cbb-4f6c-9d4c-be9357cd21ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:34:35.139336 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d00581a-8cbb-4f6c-9d4c-be9357cd21ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:34:35.142397 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d00581a-8cbb-4f6c-9d4c-be9357cd21ba service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:34:35.163318 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1137/2273] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:34:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:34:43.846121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:34:45.156649 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:35:03.216041 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d1b986-3b77-49e1-bf9d-b8f80f933334 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:35:03.231410 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d1b986-3b77-49e1-bf9d-b8f80f933334 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:35:03.241251 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104950a-87b1-4d61-94f6-cec2270e39e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:35:03.242275 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104950a-87b1-4d61-94f6-cec2270e39e3 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:35:03.242606 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104950a-87b1-4d61-94f6-cec2270e39e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:35:03.242945 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104950a-87b1-4d61-94f6-cec2270e39e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:35:03.252856 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104950a-87b1-4d61-94f6-cec2270e39e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:35:03.255212 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104950a-87b1-4d61-94f6-cec2270e39e3 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:35:03.299007 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1137/2274] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:35:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:35:13.294870 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=13 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:35:58.731902 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00581a-8cbb-4f6c-9d4c-be9357cd21ba service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:35:58.751474 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00581a-8cbb-4f6c-9d4c-be9357cd21ba service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:35:58.759842 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890dd001-63e1-4831-bace-6c71f6a85cd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:35:58.761000 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890dd001-63e1-4831-bace-6c71f6a85cd7 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:35:58.761461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890dd001-63e1-4831-bace-6c71f6a85cd7 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:35:58.761889 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890dd001-63e1-4831-bace-6c71f6a85cd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:35:58.775896 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890dd001-63e1-4831-bace-6c71f6a85cd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:35:58.777220 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890dd001-63e1-4831-bace-6c71f6a85cd7 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:35:58.811802 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1138/2275] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:35:58 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:36:07.436501 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104950a-87b1-4d61-94f6-cec2270e39e3 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:36:07.446948 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104950a-87b1-4d61-94f6-cec2270e39e3 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:36:07.450789 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ef59a-682b-4657-8404-a53ce9328bd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:36:07.451468 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ef59a-682b-4657-8404-a53ce9328bd5 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:36:07.451799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ef59a-682b-4657-8404-a53ce9328bd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:36:07.452084 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ef59a-682b-4657-8404-a53ce9328bd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:36:07.462687 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989ef59a-682b-4657-8404-a53ce9328bd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:36:07.464280 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989ef59a-682b-4657-8404-a53ce9328bd5 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:36:07.500359 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1138/2276] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:36:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:36:08.808490 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=13 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:36:17.497382 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=13 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:36:20.989452 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890dd001-63e1-4831-bace-6c71f6a85cd7 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:36:21.006062 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890dd001-63e1-4831-bace-6c71f6a85cd7 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:36:21.010795 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a998d4a-7038-4a4a-ada9-f68b2befdc33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:36:21.011483 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a998d4a-7038-4a4a-ada9-f68b2befdc33 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:36:21.011807 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a998d4a-7038-4a4a-ada9-f68b2befdc33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:36:21.012092 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a998d4a-7038-4a4a-ada9-f68b2befdc33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:36:21.022290 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a998d4a-7038-4a4a-ada9-f68b2befdc33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:36:21.024134 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a998d4a-7038-4a4a-ada9-f68b2befdc33 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:36:21.034940 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1139/2277] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:36:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:36:31.029940 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:38:07.893650 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ef59a-682b-4657-8404-a53ce9328bd5 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:38:07.910882 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ef59a-682b-4657-8404-a53ce9328bd5 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:38:07.917320 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a003782-3812-4ad8-ab66-ad99f3d98ef6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:38:07.918429 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a003782-3812-4ad8-ab66-ad99f3d98ef6 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:38:07.918920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a003782-3812-4ad8-ab66-ad99f3d98ef6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:38:07.919252 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a003782-3812-4ad8-ab66-ad99f3d98ef6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:38:07.934291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a003782-3812-4ad8-ab66-ad99f3d98ef6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:38:07.936767 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a003782-3812-4ad8-ab66-ad99f3d98ef6 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:38:07.991699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1139/2278] 10.222.0.18 () {60 vars in 1396 bytes} [Mon Jul 25 19:38:07 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:38:08.048499 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a998d4a-7038-4a4a-ada9-f68b2befdc33 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:38:08.065351 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a998d4a-7038-4a4a-ada9-f68b2befdc33 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:38:08.072054 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17c7f8-def9-4347-b6aa-d55a1ecce676 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:38:08.072736 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17c7f8-def9-4347-b6aa-d55a1ecce676 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:38:08.073041 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17c7f8-def9-4347-b6aa-d55a1ecce676 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:38:08.073289 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17c7f8-def9-4347-b6aa-d55a1ecce676 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:38:08.084113 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b17c7f8-def9-4347-b6aa-d55a1ecce676 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:38:08.085604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b17c7f8-def9-4347-b6aa-d55a1ecce676 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:38:08.134695 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1140/2279] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:38:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:38:17.988369 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=13 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:38:18.128089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=13 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:38:24.401516 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a003782-3812-4ad8-ab66-ad99f3d98ef6 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:38:24.403757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2a003782-3812-4ad8-ab66-ad99f3d98ef6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:38:24.432655 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a003782-3812-4ad8-ab66-ad99f3d98ef6 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:38:24.442148 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12f691-6491-4c26-ae01-42560d7b56b3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0422d6695bf14925bf0f993cb16e64ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:38:24.443304 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12f691-6491-4c26-ae01-42560d7b56b3 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:38:24.443865 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12f691-6491-4c26-ae01-42560d7b56b3 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:38:24.444382 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12f691-6491-4c26-ae01-42560d7b56b3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:38:24.462699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe12f691-6491-4c26-ae01-42560d7b56b3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:38:24.466066 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe12f691-6491-4c26-ae01-42560d7b56b3 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:38:24.486648 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1140/2280] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:38:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:38:34.478004 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:38:50.170274 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17c7f8-def9-4347-b6aa-d55a1ecce676 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:38:50.185158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17c7f8-def9-4347-b6aa-d55a1ecce676 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:38:50.190363 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950e27b-851b-432e-9a2d-d56e8f0b5703 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:38:50.191244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950e27b-851b-432e-9a2d-d56e8f0b5703 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:38:50.191778 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950e27b-851b-432e-9a2d-d56e8f0b5703 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:38:50.192130 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950e27b-851b-432e-9a2d-d56e8f0b5703 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:38:50.194673 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e950e27b-851b-432e-9a2d-d56e8f0b5703 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:38:50.242734 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12f691-6491-4c26-ae01-42560d7b56b3 service placement] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:38:50.313500 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e950e27b-851b-432e-9a2d-d56e8f0b5703 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:38:50.316602 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e950e27b-851b-432e-9a2d-d56e8f0b5703 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:38:50.346853 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1141/2281] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:38:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:38:50.364466 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12f691-6491-4c26-ae01-42560d7b56b3 service placement] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:38:50.373693 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2db173-2efb-4fee-a0c2-17aaf8417072 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:38:50.374602 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2db173-2efb-4fee-a0c2-17aaf8417072 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:38:50.375071 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2db173-2efb-4fee-a0c2-17aaf8417072 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:38:50.376415 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2db173-2efb-4fee-a0c2-17aaf8417072 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:38:50.389886 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2db173-2efb-4fee-a0c2-17aaf8417072 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:38:50.392698 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2db173-2efb-4fee-a0c2-17aaf8417072 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:38:50.403259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1141/2282] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:38:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:38:52.013741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950e27b-851b-432e-9a2d-d56e8f0b5703 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:38:52.030541 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950e27b-851b-432e-9a2d-d56e8f0b5703 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:38:52.038405 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc661a31-8c2b-4c5b-82a8-3c905d7b0d20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:38:52.039218 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc661a31-8c2b-4c5b-82a8-3c905d7b0d20 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:38:52.039628 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc661a31-8c2b-4c5b-82a8-3c905d7b0d20 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:38:52.039983 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc661a31-8c2b-4c5b-82a8-3c905d7b0d20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:38:52.042436 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-dc661a31-8c2b-4c5b-82a8-3c905d7b0d20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:38:52.058207 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc661a31-8c2b-4c5b-82a8-3c905d7b0d20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:38:52.060094 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc661a31-8c2b-4c5b-82a8-3c905d7b0d20 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:38:52.074547 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1142/2283] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:38:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:39:00.398453 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=34 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:39:02.068335 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=47 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:40:08.652479 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2db173-2efb-4fee-a0c2-17aaf8417072 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:40:08.654116 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5a2db173-2efb-4fee-a0c2-17aaf8417072 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:40:08.734869 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2db173-2efb-4fee-a0c2-17aaf8417072 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:40:08.743818 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd3086-5741-466b-826b-160265cf8041 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:40:08.745358 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd3086-5741-466b-826b-160265cf8041 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:40:08.746126 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd3086-5741-466b-826b-160265cf8041 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:40:08.746899 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd3086-5741-466b-826b-160265cf8041 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:40:08.749489 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7ffd3086-5741-466b-826b-160265cf8041 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:40:08.773408 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd3086-5741-466b-826b-160265cf8041 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:40:08.776246 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd3086-5741-466b-826b-160265cf8041 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:40:08.801054 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1142/2284] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:40:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:40:08.861094 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc661a31-8c2b-4c5b-82a8-3c905d7b0d20 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:40:08.871538 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc661a31-8c2b-4c5b-82a8-3c905d7b0d20 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:40:08.876660 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94260c8e-ed40-4738-959f-d971b889aba5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:40:08.877330 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94260c8e-ed40-4738-959f-d971b889aba5 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:40:08.877614 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94260c8e-ed40-4738-959f-d971b889aba5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:40:08.877915 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94260c8e-ed40-4738-959f-d971b889aba5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:40:08.880428 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-94260c8e-ed40-4738-959f-d971b889aba5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:40:08.970823 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94260c8e-ed40-4738-959f-d971b889aba5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:40:08.972504 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94260c8e-ed40-4738-959f-d971b889aba5 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:40:08.985187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1143/2285] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:40:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:40:09.092239 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd3086-5741-466b-826b-160265cf8041 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:40:09.107637 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd3086-5741-466b-826b-160265cf8041 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:40:09.112613 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8e8b6-9789-4003-835d-bb27df64a2e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:40:09.113554 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8e8b6-9789-4003-835d-bb27df64a2e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:40:09.113936 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8e8b6-9789-4003-835d-bb27df64a2e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:40:09.114136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8e8b6-9789-4003-835d-bb27df64a2e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:40:09.116666 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e7d8e8b6-9789-4003-835d-bb27df64a2e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:40:09.132955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d8e8b6-9789-4003-835d-bb27df64a2e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:40:09.134433 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d8e8b6-9789-4003-835d-bb27df64a2e3 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:40:09.148139 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1143/2286] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:40:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:40:18.979749 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=33 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:40:19.142156 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=32 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:41:06.786304 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94260c8e-ed40-4738-959f-d971b889aba5 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:41:06.788698 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-94260c8e-ed40-4738-959f-d971b889aba5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:41:06.883849 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94260c8e-ed40-4738-959f-d971b889aba5 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:41:06.891406 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaa3501-7fd0-47ee-b2f0-61a78173ba1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:41:06.892305 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa3501-7fd0-47ee-b2f0-61a78173ba1b service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:41:06.892773 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa3501-7fd0-47ee-b2f0-61a78173ba1b service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:41:06.893158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa3501-7fd0-47ee-b2f0-61a78173ba1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:41:06.895436 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7aaa3501-7fd0-47ee-b2f0-61a78173ba1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:41:06.978684 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaa3501-7fd0-47ee-b2f0-61a78173ba1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:41:06.980387 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaa3501-7fd0-47ee-b2f0-61a78173ba1b service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:41:06.991738 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1144/2287] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:41:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:41:16.986830 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=55 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:41:26.021085 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8e8b6-9789-4003-835d-bb27df64a2e3 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:41:26.023668 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e7d8e8b6-9789-4003-835d-bb27df64a2e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:41:26.181729 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8e8b6-9789-4003-835d-bb27df64a2e3 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:41:26.186586 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d9878-d6f3-4ce2-859a-d730e384c08c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:41:26.187380 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d9878-d6f3-4ce2-859a-d730e384c08c service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:41:26.187765 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d9878-d6f3-4ce2-859a-d730e384c08c service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:41:26.188043 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d9878-d6f3-4ce2-859a-d730e384c08c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:41:26.190331 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f29d9878-d6f3-4ce2-859a-d730e384c08c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:41:26.202404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29d9878-d6f3-4ce2-859a-d730e384c08c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:41:26.204304 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29d9878-d6f3-4ce2-859a-d730e384c08c service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:41:26.213933 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1144/2288] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:41:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:41:36.210423 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=48 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:42:09.503206 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaa3501-7fd0-47ee-b2f0-61a78173ba1b service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:42:09.514722 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaa3501-7fd0-47ee-b2f0-61a78173ba1b service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:42:09.520822 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b35333-0ed6-447f-af0e-31010a6c3db2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:42:09.521955 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b35333-0ed6-447f-af0e-31010a6c3db2 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:42:09.522448 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b35333-0ed6-447f-af0e-31010a6c3db2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:42:09.522941 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b35333-0ed6-447f-af0e-31010a6c3db2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:42:09.526346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d9b35333-0ed6-447f-af0e-31010a6c3db2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:42:09.643304 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b35333-0ed6-447f-af0e-31010a6c3db2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:42:09.645530 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b35333-0ed6-447f-af0e-31010a6c3db2 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:42:09.659033 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1145/2289] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:42:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:42:13.947649 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d9878-d6f3-4ce2-859a-d730e384c08c service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:42:13.964352 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d9878-d6f3-4ce2-859a-d730e384c08c service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:42:13.972398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27bd74-5700-480d-8f50-4056edc5f765 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:42:13.973642 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27bd74-5700-480d-8f50-4056edc5f765 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:42:13.974789 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27bd74-5700-480d-8f50-4056edc5f765 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:42:13.975367 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27bd74-5700-480d-8f50-4056edc5f765 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:42:13.977798 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-dc27bd74-5700-480d-8f50-4056edc5f765 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:42:13.999501 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc27bd74-5700-480d-8f50-4056edc5f765 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:42:14.001308 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc27bd74-5700-480d-8f50-4056edc5f765 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:42:14.017820 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1145/2290] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:42:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:42:19.652789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=33 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:42:24.009784 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:43:08.335010 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b35333-0ed6-447f-af0e-31010a6c3db2 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:43:08.337527 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d9b35333-0ed6-447f-af0e-31010a6c3db2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:43:08.431504 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b35333-0ed6-447f-af0e-31010a6c3db2 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:43:08.441346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449f945-8d51-4e34-a493-a18276b18fd6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:43:08.442168 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449f945-8d51-4e34-a493-a18276b18fd6 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:43:08.442559 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449f945-8d51-4e34-a493-a18276b18fd6 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:43:08.442943 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449f945-8d51-4e34-a493-a18276b18fd6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:43:08.445620 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6449f945-8d51-4e34-a493-a18276b18fd6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:43:08.541920 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6449f945-8d51-4e34-a493-a18276b18fd6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:43:08.544253 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6449f945-8d51-4e34-a493-a18276b18fd6 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:43:08.555756 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1146/2291] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:43:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:43:18.550450 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=45 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:43:29.452286 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27bd74-5700-480d-8f50-4056edc5f765 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:43:29.537241 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27bd74-5700-480d-8f50-4056edc5f765 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:43:29.543717 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57657843-8651-4fcd-bfa5-2faa7b912302 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0422d6695bf14925bf0f993cb16e64ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:43:29.544407 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57657843-8651-4fcd-bfa5-2faa7b912302 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:43:29.544698 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57657843-8651-4fcd-bfa5-2faa7b912302 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:43:29.544980 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57657843-8651-4fcd-bfa5-2faa7b912302 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:43:29.547176 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-57657843-8651-4fcd-bfa5-2faa7b912302 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:43:29.583580 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57657843-8651-4fcd-bfa5-2faa7b912302 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:43:29.585320 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57657843-8651-4fcd-bfa5-2faa7b912302 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:43:29.600109 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1146/2292] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:43:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:43:39.593228 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=41 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:43:50.022461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449f945-8d51-4e34-a493-a18276b18fd6 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:43:50.037169 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449f945-8d51-4e34-a493-a18276b18fd6 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:43:50.043585 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9fc892-afce-4aad-8f0f-d10e13429dd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:43:50.044448 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fc892-afce-4aad-8f0f-d10e13429dd2 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:43:50.044859 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fc892-afce-4aad-8f0f-d10e13429dd2 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:43:50.045149 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fc892-afce-4aad-8f0f-d10e13429dd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:43:50.054622 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9fc892-afce-4aad-8f0f-d10e13429dd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:43:50.056318 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9fc892-afce-4aad-8f0f-d10e13429dd2 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:43:50.070205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1147/2293] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:43:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:44:00.068639 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:44:10.441580 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57657843-8651-4fcd-bfa5-2faa7b912302 service placement] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:44:10.469735 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57657843-8651-4fcd-bfa5-2faa7b912302 service placement] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:44:10.473438 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fcc1b9-7331-4954-a0e2-5a40d2efb710 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:44:10.474091 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fcc1b9-7331-4954-a0e2-5a40d2efb710 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:44:10.474373 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fcc1b9-7331-4954-a0e2-5a40d2efb710 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:44:10.474647 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fcc1b9-7331-4954-a0e2-5a40d2efb710 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:44:10.476643 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a7fcc1b9-7331-4954-a0e2-5a40d2efb710 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:44:10.563030 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fcc1b9-7331-4954-a0e2-5a40d2efb710 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:44:10.565966 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fcc1b9-7331-4954-a0e2-5a40d2efb710 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:44:10.583592 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1147/2294] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:44:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:44:20.575432 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=37 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:44:52.458488 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9fc892-afce-4aad-8f0f-d10e13429dd2 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:44:52.461208 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0c9fc892-afce-4aad-8f0f-d10e13429dd2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:44:52.482771 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9fc892-afce-4aad-8f0f-d10e13429dd2 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:44:52.488909 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f2fe1-c255-4ee5-9e8e-8c99e6940519 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:44:52.490053 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f2fe1-c255-4ee5-9e8e-8c99e6940519 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:44:52.490446 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f2fe1-c255-4ee5-9e8e-8c99e6940519 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:44:52.490907 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f2fe1-c255-4ee5-9e8e-8c99e6940519 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:44:52.494527 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5d3f2fe1-c255-4ee5-9e8e-8c99e6940519 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:44:52.602913 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f2fe1-c255-4ee5-9e8e-8c99e6940519 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:44:52.604657 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f2fe1-c255-4ee5-9e8e-8c99e6940519 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:44:52.636289 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1148/2295] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:44:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:44:56.192518 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157d376-8de6-4da4-a25a-1dec55a60021 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:44:56.193569 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157d376-8de6-4da4-a25a-1dec55a60021 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:44:56.194021 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157d376-8de6-4da4-a25a-1dec55a60021 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:44:56.196087 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1148/2296] 10.222.0.18 () {54 vars in 921 bytes} [Mon Jul 25 19:44:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 19:44:56.243989 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ffa7a-a818-4dbf-9dce-f26bde77320c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:44:56.244635 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ffa7a-a818-4dbf-9dce-f26bde77320c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:44:56.245227 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ffa7a-a818-4dbf-9dce-f26bde77320c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:44:56.335255 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-097ffa7a-a818-4dbf-9dce-f26bde77320c None None] MFA Rules not processed for user `0422d6695bf14925bf0f993cb16e64ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:44:56.336589 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-097ffa7a-a818-4dbf-9dce-f26bde77320c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:44:56.347177 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1149/2297] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Jul 25 19:44:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:44:56.391030 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157d376-8de6-4da4-a25a-1dec55a60021 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:44:56.405656 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157d376-8de6-4da4-a25a-1dec55a60021 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:44:56.410818 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb77aa-845e-4dbd-a7b9-772b029828b4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0422d6695bf14925bf0f993cb16e64ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:44:56.411485 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb77aa-845e-4dbd-a7b9-772b029828b4 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:44:56.411937 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb77aa-845e-4dbd-a7b9-772b029828b4 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:44:56.412219 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb77aa-845e-4dbd-a7b9-772b029828b4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:44:56.422544 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eb77aa-845e-4dbd-a7b9-772b029828b4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:44:56.424092 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eb77aa-845e-4dbd-a7b9-772b029828b4 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:44:56.435376 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1149/2298] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:44:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:45:00.310323 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ffa7a-a818-4dbf-9dce-f26bde77320c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:45:00.312303 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-097ffa7a-a818-4dbf-9dce-f26bde77320c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:45:00.329369 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ffa7a-a818-4dbf-9dce-f26bde77320c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:45:00.334983 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e940f-2a20-4e3a-8e50-a19d901fe91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:45:00.335913 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e940f-2a20-4e3a-8e50-a19d901fe91d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:45:00.336224 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e940f-2a20-4e3a-8e50-a19d901fe91d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:45:00.336502 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e940f-2a20-4e3a-8e50-a19d901fe91d admin admin] PATH_INFO: `/v3/users/574d0113dce74ab199c0162abe26934b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:45:00.338589 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724e940f-2a20-4e3a-8e50-a19d901fe91d admin admin] RBAC: Authorizing `identity:delete_user(user_id=574d0113dce74ab199c0162abe26934b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:45:00.340665 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724e940f-2a20-4e3a-8e50-a19d901fe91d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:45:00.405457 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-724e940f-2a20-4e3a-8e50-a19d901fe91d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '574d0113dce74ab199c0162abe26934b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:45:00.414158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-724e940f-2a20-4e3a-8e50-a19d901fe91d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '574d0113dce74ab199c0162abe26934b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:45:00.423992 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-724e940f-2a20-4e3a-8e50-a19d901fe91d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '574d0113dce74ab199c0162abe26934b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:45:00.424766 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-724e940f-2a20-4e3a-8e50-a19d901fe91d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '574d0113dce74ab199c0162abe26934b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:45:00.438805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-724e940f-2a20-4e3a-8e50-a19d901fe91d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '574d0113dce74ab199c0162abe26934b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:45:00.441157 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:45:00.447503 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1150/2299] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:45:00 2022] DELETE /identity/v3/users/574d0113dce74ab199c0162abe26934b => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:45:00.690788 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb77aa-845e-4dbd-a7b9-772b029828b4 service placement] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:45:00.730816 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb77aa-845e-4dbd-a7b9-772b029828b4 service placement] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:45:00.818478 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aea0de-c417-4314-9dc2-1efc1b84c169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:45:00.820074 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aea0de-c417-4314-9dc2-1efc1b84c169 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:45:00.820579 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aea0de-c417-4314-9dc2-1efc1b84c169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:45:00.821163 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aea0de-c417-4314-9dc2-1efc1b84c169 admin admin] PATH_INFO: `/v3/projects/9166cf61c5d944fd9f1e2174c6544faa` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:45:00.823545 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aea0de-c417-4314-9dc2-1efc1b84c169 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9166cf61c5d944fd9f1e2174c6544faa)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:45:00.827610 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aea0de-c417-4314-9dc2-1efc1b84c169 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:45:00.893202 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d4aea0de-c417-4314-9dc2-1efc1b84c169 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9166cf61c5d944fd9f1e2174c6544faa', 'request_id': 'req-d4aea0de-c417-4314-9dc2-1efc1b84c169'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:45:00.901769 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d4aea0de-c417-4314-9dc2-1efc1b84c169 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9166cf61c5d944fd9f1e2174c6544faa', 'request_id': 'req-d4aea0de-c417-4314-9dc2-1efc1b84c169'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:45:00.908577 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d4aea0de-c417-4314-9dc2-1efc1b84c169 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9166cf61c5d944fd9f1e2174c6544faa', 'request_id': 'req-d4aea0de-c417-4314-9dc2-1efc1b84c169'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:45:00.924202 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d4aea0de-c417-4314-9dc2-1efc1b84c169 admin admin] The token cache is being invalidate because project 9166cf61c5d944fd9f1e2174c6544faa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:45:00.924475 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d4aea0de-c417-4314-9dc2-1efc1b84c169 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:45:00.925125 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d4aea0de-c417-4314-9dc2-1efc1b84c169 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:45:00.926737 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1150/2300] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:45:00 2022] DELETE /identity/v3/projects/9166cf61c5d944fd9f1e2174c6544faa => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:45:00.931111 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e940f-2a20-4e3a-8e50-a19d901fe91d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:45:00.932935 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-724e940f-2a20-4e3a-8e50-a19d901fe91d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:45:00.948361 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e940f-2a20-4e3a-8e50-a19d901fe91d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:45:01.029301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0d2be-4734-4763-b3cd-26f409c84aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:45:01.030225 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0d2be-4734-4763-b3cd-26f409c84aa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:45:01.030509 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0d2be-4734-4763-b3cd-26f409c84aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:45:01.030794 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0d2be-4734-4763-b3cd-26f409c84aa4 admin admin] PATH_INFO: `/v3/users/064f1f93ebd44f4e89c3fb52322e42bb` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:45:01.048767 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf0d2be-4734-4763-b3cd-26f409c84aa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=064f1f93ebd44f4e89c3fb52322e42bb)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:45:01.052171 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf0d2be-4734-4763-b3cd-26f409c84aa4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:45:01.115101 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fcf0d2be-4734-4763-b3cd-26f409c84aa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '064f1f93ebd44f4e89c3fb52322e42bb'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:45:01.125311 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fcf0d2be-4734-4763-b3cd-26f409c84aa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '064f1f93ebd44f4e89c3fb52322e42bb'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:45:01.133323 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fcf0d2be-4734-4763-b3cd-26f409c84aa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '064f1f93ebd44f4e89c3fb52322e42bb'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:45:01.134687 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fcf0d2be-4734-4763-b3cd-26f409c84aa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '064f1f93ebd44f4e89c3fb52322e42bb'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:45:01.148735 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fcf0d2be-4734-4763-b3cd-26f409c84aa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '064f1f93ebd44f4e89c3fb52322e42bb'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:45:01.156444 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1151/2301] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:45:00 2022] DELETE /identity/v3/users/064f1f93ebd44f4e89c3fb52322e42bb => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:45:01.468170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aea0de-c417-4314-9dc2-1efc1b84c169 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:45:01.470369 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d4aea0de-c417-4314-9dc2-1efc1b84c169 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:45:01.504014 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aea0de-c417-4314-9dc2-1efc1b84c169 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:45:01.594386 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6912ae41-f608-4b71-aedf-ac2196efbefa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:45:01.595627 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912ae41-f608-4b71-aedf-ac2196efbefa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:45:01.596124 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912ae41-f608-4b71-aedf-ac2196efbefa admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:45:01.596611 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912ae41-f608-4b71-aedf-ac2196efbefa admin admin] PATH_INFO: `/v3/projects/e70d838c7e8b416791220cf1643b545e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:45:01.619472 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6912ae41-f608-4b71-aedf-ac2196efbefa admin admin] RBAC: Authorizing `identity:delete_project(project_id=e70d838c7e8b416791220cf1643b545e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:45:01.623529 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6912ae41-f608-4b71-aedf-ac2196efbefa admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:45:01.645888 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:45:01.700222 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6912ae41-f608-4b71-aedf-ac2196efbefa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e70d838c7e8b416791220cf1643b545e', 'request_id': 'req-6912ae41-f608-4b71-aedf-ac2196efbefa'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:45:01.707736 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6912ae41-f608-4b71-aedf-ac2196efbefa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e70d838c7e8b416791220cf1643b545e', 'request_id': 'req-6912ae41-f608-4b71-aedf-ac2196efbefa'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:45:01.714871 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6912ae41-f608-4b71-aedf-ac2196efbefa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e70d838c7e8b416791220cf1643b545e', 'request_id': 'req-6912ae41-f608-4b71-aedf-ac2196efbefa'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:45:01.729995 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6912ae41-f608-4b71-aedf-ac2196efbefa admin admin] The token cache is being invalidate because project e70d838c7e8b416791220cf1643b545e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:45:01.730362 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6912ae41-f608-4b71-aedf-ac2196efbefa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:45:01.731493 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6912ae41-f608-4b71-aedf-ac2196efbefa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:45:01.733064 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1151/2302] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:45:01 2022] DELETE /identity/v3/projects/e70d838c7e8b416791220cf1643b545e => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:45:08.671363 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0d2be-4734-4763-b3cd-26f409c84aa4 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:45:08.675630 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fcf0d2be-4734-4763-b3cd-26f409c84aa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:45:08.770434 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0d2be-4734-4763-b3cd-26f409c84aa4 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:45:08.775697 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:45:08.853018 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f406c-182f-4ee4-b223-7205d459c231 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:45:08.853907 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f406c-182f-4ee4-b223-7205d459c231 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:45:08.854194 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f406c-182f-4ee4-b223-7205d459c231 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:45:08.854469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f406c-182f-4ee4-b223-7205d459c231 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:45:08.856499 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c88f406c-182f-4ee4-b223-7205d459c231 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:45:08.938166 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88f406c-182f-4ee4-b223-7205d459c231 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:45:08.939956 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88f406c-182f-4ee4-b223-7205d459c231 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:45:08.962231 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1152/2303] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:45:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:45:09.054949 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6912ae41-f608-4b71-aedf-ac2196efbefa admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:45:09.057210 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6912ae41-f608-4b71-aedf-ac2196efbefa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:45:09.120971 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6912ae41-f608-4b71-aedf-ac2196efbefa admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:45:09.124831 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba3c64-8088-420f-82a8-bcf8f9d41705 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:45:09.125496 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba3c64-8088-420f-82a8-bcf8f9d41705 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:45:09.125784 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba3c64-8088-420f-82a8-bcf8f9d41705 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:45:09.126095 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba3c64-8088-420f-82a8-bcf8f9d41705 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:45:09.128106 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-53ba3c64-8088-420f-82a8-bcf8f9d41705 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:45:09.144722 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ba3c64-8088-420f-82a8-bcf8f9d41705 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:45:09.146207 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ba3c64-8088-420f-82a8-bcf8f9d41705 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:45:09.159562 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1152/2304] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:45:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:45:09.694493 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f406c-182f-4ee4-b223-7205d459c231 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:45:09.705004 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f406c-182f-4ee4-b223-7205d459c231 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:45:09.708696 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d6177-5330-4635-82df-07d16c01fd0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:45:09.709355 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d6177-5330-4635-82df-07d16c01fd0a service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:45:09.709643 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d6177-5330-4635-82df-07d16c01fd0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:45:09.709925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d6177-5330-4635-82df-07d16c01fd0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:45:09.712337 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-278d6177-5330-4635-82df-07d16c01fd0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:45:09.728202 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278d6177-5330-4635-82df-07d16c01fd0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:45:09.729712 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278d6177-5330-4635-82df-07d16c01fd0a service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:45:09.743290 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1153/2305] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:45:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:45:19.154258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:45:19.744707 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=67 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:46:29.977931 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba3c64-8088-420f-82a8-bcf8f9d41705 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:46:29.988631 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba3c64-8088-420f-82a8-bcf8f9d41705 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:46:29.993255 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95762950-52c3-438e-8685-4bfadb7e78d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:46:29.993848 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95762950-52c3-438e-8685-4bfadb7e78d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:46:29.994131 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95762950-52c3-438e-8685-4bfadb7e78d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:46:29.994614 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95762950-52c3-438e-8685-4bfadb7e78d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:46:29.996426 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-95762950-52c3-438e-8685-4bfadb7e78d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:46:30.009519 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95762950-52c3-438e-8685-4bfadb7e78d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:46:30.011180 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95762950-52c3-438e-8685-4bfadb7e78d2 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:46:30.020530 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1153/2306] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:46:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:46:40.016324 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:46:51.036807 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d6177-5330-4635-82df-07d16c01fd0a service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:46:51.038407 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-278d6177-5330-4635-82df-07d16c01fd0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:46:51.141808 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d6177-5330-4635-82df-07d16c01fd0a service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:46:51.149216 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a14cdba-4acd-46d0-977c-e9410a2e0861 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:46:51.151503 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14cdba-4acd-46d0-977c-e9410a2e0861 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:46:51.151503 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14cdba-4acd-46d0-977c-e9410a2e0861 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:46:51.151503 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14cdba-4acd-46d0-977c-e9410a2e0861 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:46:51.154774 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2a14cdba-4acd-46d0-977c-e9410a2e0861 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:46:51.275459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a14cdba-4acd-46d0-977c-e9410a2e0861 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:46:51.279760 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a14cdba-4acd-46d0-977c-e9410a2e0861 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:46:51.313765 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1154/2307] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:46:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:47:01.308500 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=61 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:47:10.347160 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95762950-52c3-438e-8685-4bfadb7e78d2 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:47:10.348779 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-95762950-52c3-438e-8685-4bfadb7e78d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:47:10.376361 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95762950-52c3-438e-8685-4bfadb7e78d2 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:47:10.384506 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c3caee-7398-486b-852c-222c48a851b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:47:10.385170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c3caee-7398-486b-852c-222c48a851b3 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:47:10.385462 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c3caee-7398-486b-852c-222c48a851b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:47:10.385740 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c3caee-7398-486b-852c-222c48a851b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:47:10.387689 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-31c3caee-7398-486b-852c-222c48a851b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:47:10.472753 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c3caee-7398-486b-852c-222c48a851b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:47:10.474437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c3caee-7398-486b-852c-222c48a851b3 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:47:10.505263 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1154/2308] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:47:10 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:47:20.493906 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=36 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:48:10.751721 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a14cdba-4acd-46d0-977c-e9410a2e0861 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:48:10.753537 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2a14cdba-4acd-46d0-977c-e9410a2e0861 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:48:10.766864 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a14cdba-4acd-46d0-977c-e9410a2e0861 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:48:10.771174 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6bdb63-3f0f-4533-a2d1-76f9ed753164 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:48:10.771950 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6bdb63-3f0f-4533-a2d1-76f9ed753164 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:48:10.772351 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6bdb63-3f0f-4533-a2d1-76f9ed753164 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:48:10.772752 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6bdb63-3f0f-4533-a2d1-76f9ed753164 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:48:10.774762 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-af6bdb63-3f0f-4533-a2d1-76f9ed753164 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:48:10.862783 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6bdb63-3f0f-4533-a2d1-76f9ed753164 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:48:10.864282 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6bdb63-3f0f-4533-a2d1-76f9ed753164 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:48:10.886532 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1155/2309] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:48:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:48:13.621470 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809e5c2-13ae-4d3f-8521-37f4bace0dad None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:48:13.622312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809e5c2-13ae-4d3f-8521-37f4bace0dad None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:48:13.622312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809e5c2-13ae-4d3f-8521-37f4bace0dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:48:13.670646 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9809e5c2-13ae-4d3f-8521-37f4bace0dad None None] MFA Rules not processed for user `6ecf34ceee294b53b5be7c1723b42950`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:48:13.673384 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9809e5c2-13ae-4d3f-8521-37f4bace0dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:48:13.688328 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1155/2310] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:48:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:48:14.055161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6bdb63-3f0f-4533-a2d1-76f9ed753164 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:48:14.066360 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6bdb63-3f0f-4533-a2d1-76f9ed753164 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:48:14.070730 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c1c5a-fd82-4c9c-a0de-d5d9d4992e9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:48:14.073205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c1c5a-fd82-4c9c-a0de-d5d9d4992e9c service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:48:14.073205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c1c5a-fd82-4c9c-a0de-d5d9d4992e9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:48:14.073205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c1c5a-fd82-4c9c-a0de-d5d9d4992e9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:48:14.074573 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-588c1c5a-fd82-4c9c-a0de-d5d9d4992e9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:48:14.090283 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588c1c5a-fd82-4c9c-a0de-d5d9d4992e9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:48:14.094460 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588c1c5a-fd82-4c9c-a0de-d5d9d4992e9c service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:48:14.102071 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1156/2311] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:48:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:48:15.584670 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9809e5c2-13ae-4d3f-8521-37f4bace0dad None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:48:15.585932 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9809e5c2-13ae-4d3f-8521-37f4bace0dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:48:15.710664 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9809e5c2-13ae-4d3f-8521-37f4bace0dad None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:48:15.716314 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f0dd1-1462-466d-875b-5f60858aca66 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0422d6695bf14925bf0f993cb16e64ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:48:15.717453 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f0dd1-1462-466d-875b-5f60858aca66 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:48:15.717786 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f0dd1-1462-466d-875b-5f60858aca66 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:48:15.718120 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f0dd1-1462-466d-875b-5f60858aca66 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:48:15.720413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-301f0dd1-1462-466d-875b-5f60858aca66 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:48:15.736376 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f0dd1-1462-466d-875b-5f60858aca66 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:48:15.738385 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f0dd1-1462-466d-875b-5f60858aca66 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:48:15.753194 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1156/2312] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:48:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:48:21.979608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c1c5a-fd82-4c9c-a0de-d5d9d4992e9c service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:48:21.983558 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-588c1c5a-fd82-4c9c-a0de-d5d9d4992e9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:48:22.008844 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c1c5a-fd82-4c9c-a0de-d5d9d4992e9c service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:48:22.014736 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3380cf-8ba1-43db-bad1-3ce9b26ef135 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:48:22.015566 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3380cf-8ba1-43db-bad1-3ce9b26ef135 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:48:22.016690 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3380cf-8ba1-43db-bad1-3ce9b26ef135 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:48:22.016690 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3380cf-8ba1-43db-bad1-3ce9b26ef135 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:48:22.018736 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5f3380cf-8ba1-43db-bad1-3ce9b26ef135 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:48:22.036600 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3380cf-8ba1-43db-bad1-3ce9b26ef135 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:48:22.039688 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3380cf-8ba1-43db-bad1-3ce9b26ef135 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:48:22.055068 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1157/2313] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:48:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:48:25.745919 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=38 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:48:28.306434 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f0dd1-1462-466d-875b-5f60858aca66 service placement] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:48:28.320681 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f0dd1-1462-466d-875b-5f60858aca66 service placement] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:48:28.326604 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbb13ce-9497-47cb-8b02-e5d4a22a8c25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:48:28.327287 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb13ce-9497-47cb-8b02-e5d4a22a8c25 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:48:28.327627 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb13ce-9497-47cb-8b02-e5d4a22a8c25 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:48:28.327890 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb13ce-9497-47cb-8b02-e5d4a22a8c25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:48:28.330024 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0bbb13ce-9497-47cb-8b02-e5d4a22a8c25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:48:28.344112 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbb13ce-9497-47cb-8b02-e5d4a22a8c25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:48:28.346325 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbb13ce-9497-47cb-8b02-e5d4a22a8c25 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:48:28.360029 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1157/2314] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:48:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:48:32.048364 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=48 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:48:35.475601 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3380cf-8ba1-43db-bad1-3ce9b26ef135 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:48:35.487820 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3380cf-8ba1-43db-bad1-3ce9b26ef135 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:48:35.492931 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd8f937-18f5-4deb-867d-e80c1c5f4835 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0422d6695bf14925bf0f993cb16e64ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:48:35.493586 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8f937-18f5-4deb-867d-e80c1c5f4835 service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:48:35.493866 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8f937-18f5-4deb-867d-e80c1c5f4835 service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:48:35.494141 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8f937-18f5-4deb-867d-e80c1c5f4835 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:48:35.495964 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1bd8f937-18f5-4deb-867d-e80c1c5f4835 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:48:35.509578 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd8f937-18f5-4deb-867d-e80c1c5f4835 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:48:35.511167 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd8f937-18f5-4deb-867d-e80c1c5f4835 service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:48:35.521609 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1158/2315] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:48:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:48:38.354393 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:48:45.517562 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:49:11.127153 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbb13ce-9497-47cb-8b02-e5d4a22a8c25 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:49:11.141483 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbb13ce-9497-47cb-8b02-e5d4a22a8c25 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:49:11.148986 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2217cdb1-10c0-4333-a124-5d7ff59717bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:49:11.149763 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217cdb1-10c0-4333-a124-5d7ff59717bb service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:49:11.150240 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217cdb1-10c0-4333-a124-5d7ff59717bb service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:49:11.150634 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217cdb1-10c0-4333-a124-5d7ff59717bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:49:11.153140 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2217cdb1-10c0-4333-a124-5d7ff59717bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:49:11.245154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2217cdb1-10c0-4333-a124-5d7ff59717bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:49:11.248816 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2217cdb1-10c0-4333-a124-5d7ff59717bb service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:49:11.270500 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1158/2316] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:49:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:49:21.267415 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=36 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:50:08.518565 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd8f937-18f5-4deb-867d-e80c1c5f4835 service placement] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:50:08.523810 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2217cdb1-10c0-4333-a124-5d7ff59717bb service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:50:08.531465 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd8f937-18f5-4deb-867d-e80c1c5f4835 service placement] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:50:08.535595 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d6d7ba-d020-4aca-9aaf-558676a2077c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:50:08.535749 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2217cdb1-10c0-4333-a124-5d7ff59717bb service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:50:08.536354 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d6d7ba-d020-4aca-9aaf-558676a2077c service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:50:08.536650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d6d7ba-d020-4aca-9aaf-558676a2077c service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:50:08.536929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d6d7ba-d020-4aca-9aaf-558676a2077c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:50:08.539326 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b905b-be21-446c-863e-7c524c12c4d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:50:08.540012 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b905b-be21-446c-863e-7c524c12c4d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:50:08.540296 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b905b-be21-446c-863e-7c524c12c4d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:50:08.540570 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b905b-be21-446c-863e-7c524c12c4d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:50:08.548809 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d6d7ba-d020-4aca-9aaf-558676a2077c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:50:08.550283 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d6d7ba-d020-4aca-9aaf-558676a2077c service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:50:08.553049 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777b905b-be21-446c-863e-7c524c12c4d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:50:08.554911 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777b905b-be21-446c-863e-7c524c12c4d8 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:50:08.563413 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1159/2317] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:50:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:50:08.568619 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1159/2318] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:50:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:50:11.077802 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d6d7ba-d020-4aca-9aaf-558676a2077c service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:50:11.089215 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d6d7ba-d020-4aca-9aaf-558676a2077c service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:50:11.092962 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9802884-4249-49da-a056-7e71bf7f2390 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:50:11.093623 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9802884-4249-49da-a056-7e71bf7f2390 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:50:11.093944 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9802884-4249-49da-a056-7e71bf7f2390 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:50:11.094225 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9802884-4249-49da-a056-7e71bf7f2390 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:50:11.096109 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f9802884-4249-49da-a056-7e71bf7f2390 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:50:11.110306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-f9802884-4249-49da-a056-7e71bf7f2390 service cinder] Could not find project: 9166cf61c5d944fd9f1e2174c6544faa.: keystone.exception.ProjectNotFound: Could not find project: 9166cf61c5d944fd9f1e2174c6544faa. Jul 25 19:50:11.110306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 19:50:11.110306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 25 19:50:11.110306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 19:50:11.110306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 25 19:50:11.110306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 19:50:11.110306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 19:50:11.110306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 19:50:11.110306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 19:50:11.110306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 19:50:11.110306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 19:50:11.110306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 19:50:11.110306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Jul 25 19:50:11.110306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Jul 25 19:50:11.110306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Jul 25 19:50:11.111616 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Jul 25 19:50:11.111616 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Jul 25 19:50:11.111616 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Jul 25 19:50:11.111616 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:50:11.111616 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:50:11.111616 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Jul 25 19:50:11.111616 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Jul 25 19:50:11.111616 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:50:11.111616 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:50:11.111616 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:50:11.111616 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:50:11.111616 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:50:11.111616 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:50:11.111616 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:50:11.111616 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:50:11.111616 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:50:11.111616 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:50:11.113187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:50:11.113187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:50:11.113187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:50:11.113187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:50:11.113187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Jul 25 19:50:11.113187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Jul 25 19:50:11.113187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Jul 25 19:50:11.113187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._validate_token_resources() Jul 25 19:50:11.113187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Jul 25 19:50:11.113187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Jul 25 19:50:11.113187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Jul 25 19:50:11.113187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Jul 25 19:50:11.113187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 19:50:11.113187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 19:50:11.113187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 19:50:11.113187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 19:50:11.113187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 19:50:11.114343 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 19:50:11.114343 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 19:50:11.114343 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jul 25 19:50:11.114343 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 19:50:11.114343 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jul 25 19:50:11.114343 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 19:50:11.114343 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 19:50:11.114343 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 19:50:11.114343 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jul 25 19:50:11.114343 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 19:50:11.114343 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jul 25 19:50:11.114343 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 19:50:11.114343 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 19:50:11.114343 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 19:50:11.114343 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 19:50:11.114343 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 19:50:11.114343 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 19:50:11.115428 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 9166cf61c5d944fd9f1e2174c6544faa. Jul 25 19:50:11.115428 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jul 25 19:50:11.115428 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1160/2319] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:50:11 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 35 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Jul 25 19:50:11.154865 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b905b-be21-446c-863e-7c524c12c4d8 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:50:11.164525 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b905b-be21-446c-863e-7c524c12c4d8 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:50:11.168774 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7605d33-9657-4b8a-bd91-f2a4e6fc7992 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:50:11.169218 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7605d33-9657-4b8a-bd91-f2a4e6fc7992 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:50:11.169504 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7605d33-9657-4b8a-bd91-f2a4e6fc7992 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:50:11.169778 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7605d33-9657-4b8a-bd91-f2a4e6fc7992 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:50:11.181016 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7605d33-9657-4b8a-bd91-f2a4e6fc7992 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:50:11.182465 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7605d33-9657-4b8a-bd91-f2a4e6fc7992 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:50:11.193633 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1160/2320] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:50:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:50:21.100119 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=10 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:50:21.188354 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=12 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:50:54.485327 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9802884-4249-49da-a056-7e71bf7f2390 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:50:54.516369 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9802884-4249-49da-a056-7e71bf7f2390 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:50:54.527202 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e203ee42-55d2-4367-b7e0-a0f8b2b739c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:50:54.529141 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e203ee42-55d2-4367-b7e0-a0f8b2b739c5 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:50:54.529697 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e203ee42-55d2-4367-b7e0-a0f8b2b739c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:50:54.530169 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e203ee42-55d2-4367-b7e0-a0f8b2b739c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:50:54.533824 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e203ee42-55d2-4367-b7e0-a0f8b2b739c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:50:54.558111 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e203ee42-55d2-4367-b7e0-a0f8b2b739c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:50:54.561894 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e203ee42-55d2-4367-b7e0-a0f8b2b739c5 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:50:54.574833 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1161/2321] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:50:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:50:56.352793 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab81af2-dee3-4f93-9042-64154f19514d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:50:56.353377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab81af2-dee3-4f93-9042-64154f19514d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:50:56.353377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab81af2-dee3-4f93-9042-64154f19514d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:50:56.386349 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-eab81af2-dee3-4f93-9042-64154f19514d None None] MFA Rules not processed for user `ac2e584f88cd46f280b7fe04aa1382fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:50:56.387685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-eab81af2-dee3-4f93-9042-64154f19514d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:50:56.395106 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1161/2322] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:50:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:50:56.734299 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e203ee42-55d2-4367-b7e0-a0f8b2b739c5 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:50:56.744891 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e203ee42-55d2-4367-b7e0-a0f8b2b739c5 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:50:56.749413 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689922b6-70a2-48e4-95b4-d77d94b067b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:50:56.750274 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689922b6-70a2-48e4-95b4-d77d94b067b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:50:56.750564 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689922b6-70a2-48e4-95b4-d77d94b067b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:50:56.750847 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689922b6-70a2-48e4-95b4-d77d94b067b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:50:56.752937 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-689922b6-70a2-48e4-95b4-d77d94b067b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:50:56.766348 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689922b6-70a2-48e4-95b4-d77d94b067b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:50:56.767859 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689922b6-70a2-48e4-95b4-d77d94b067b7 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:50:56.777574 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1162/2323] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:50:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:51:06.383539 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:51:06.773675 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=12 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:51:34.983584 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab81af2-dee3-4f93-9042-64154f19514d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:51:35.028973 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab81af2-dee3-4f93-9042-64154f19514d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:51:35.033280 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc64dcd-3c42-4c48-9e12-f5f7ee673cef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:51:35.033950 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc64dcd-3c42-4c48-9e12-f5f7ee673cef service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:51:35.034242 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc64dcd-3c42-4c48-9e12-f5f7ee673cef service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:51:35.034519 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc64dcd-3c42-4c48-9e12-f5f7ee673cef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:51:35.043870 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc64dcd-3c42-4c48-9e12-f5f7ee673cef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:51:35.045317 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc64dcd-3c42-4c48-9e12-f5f7ee673cef service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:51:35.054579 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1162/2324] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:51:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:51:45.050648 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=18 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:52:11.587146 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689922b6-70a2-48e4-95b4-d77d94b067b7 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:52:11.609871 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689922b6-70a2-48e4-95b4-d77d94b067b7 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:52:11.614717 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65783247-911c-48dd-b0f4-9cb0efc6adbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:52:11.615381 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65783247-911c-48dd-b0f4-9cb0efc6adbe service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:52:11.615690 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65783247-911c-48dd-b0f4-9cb0efc6adbe service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:52:11.615971 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65783247-911c-48dd-b0f4-9cb0efc6adbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:52:11.642197 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65783247-911c-48dd-b0f4-9cb0efc6adbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:52:11.643694 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65783247-911c-48dd-b0f4-9cb0efc6adbe service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:52:11.654229 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1163/2325] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:52:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:52:11.936969 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc64dcd-3c42-4c48-9e12-f5f7ee673cef service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:52:11.947360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc64dcd-3c42-4c48-9e12-f5f7ee673cef service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:52:11.952886 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4d23a-7916-4f20-a782-3019f247c5c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:52:11.954145 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4d23a-7916-4f20-a782-3019f247c5c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:52:11.954608 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4d23a-7916-4f20-a782-3019f247c5c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:52:11.955105 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f4d23a-7916-4f20-a782-3019f247c5c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:52:11.970053 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f4d23a-7916-4f20-a782-3019f247c5c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:52:11.971532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f4d23a-7916-4f20-a782-3019f247c5c6 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:52:11.982597 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1163/2326] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:52:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:52:21.649037 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=17 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:52:21.977048 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:53:42.482565 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65783247-911c-48dd-b0f4-9cb0efc6adbe service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:53:42.514305 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65783247-911c-48dd-b0f4-9cb0efc6adbe service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:53:42.519994 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce5d0f-1264-458d-8b75-10013680364c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0422d6695bf14925bf0f993cb16e64ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:53:42.520781 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce5d0f-1264-458d-8b75-10013680364c service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:53:42.521166 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce5d0f-1264-458d-8b75-10013680364c service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:53:42.521565 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce5d0f-1264-458d-8b75-10013680364c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:53:42.532745 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fce5d0f-1264-458d-8b75-10013680364c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:53:42.534532 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fce5d0f-1264-458d-8b75-10013680364c service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:53:42.546237 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1164/2327] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:53:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:53:52.541272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=13 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:54:10.818884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4d23a-7916-4f20-a782-3019f247c5c6 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:54:10.864911 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f4d23a-7916-4f20-a782-3019f247c5c6 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:54:10.871836 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea528cb-e8f1-49e3-a01c-b3dd10046c7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:54:10.872726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea528cb-e8f1-49e3-a01c-b3dd10046c7f service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:54:10.873139 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea528cb-e8f1-49e3-a01c-b3dd10046c7f service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:54:10.873520 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea528cb-e8f1-49e3-a01c-b3dd10046c7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:54:10.888803 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea528cb-e8f1-49e3-a01c-b3dd10046c7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:54:10.891862 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea528cb-e8f1-49e3-a01c-b3dd10046c7f service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:54:10.903935 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1164/2328] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:54:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:54:12.412665 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce5d0f-1264-458d-8b75-10013680364c service placement] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:54:12.437666 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce5d0f-1264-458d-8b75-10013680364c service placement] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:54:12.441824 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642064fc-c4cf-4ce8-8358-9c39115ad1a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:54:12.442486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642064fc-c4cf-4ce8-8358-9c39115ad1a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:54:12.442779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642064fc-c4cf-4ce8-8358-9c39115ad1a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:54:12.443063 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642064fc-c4cf-4ce8-8358-9c39115ad1a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:54:12.453094 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642064fc-c4cf-4ce8-8358-9c39115ad1a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:54:12.454548 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642064fc-c4cf-4ce8-8358-9c39115ad1a9 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:54:12.467669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1165/2329] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:54:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:54:14.525533 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea528cb-e8f1-49e3-a01c-b3dd10046c7f service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:54:14.544908 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea528cb-e8f1-49e3-a01c-b3dd10046c7f service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:54:14.552584 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39943d6f-0018-47c3-bd28-13aa4048de53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:54:14.553366 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39943d6f-0018-47c3-bd28-13aa4048de53 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:54:14.553695 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39943d6f-0018-47c3-bd28-13aa4048de53 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:54:14.554194 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39943d6f-0018-47c3-bd28-13aa4048de53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:54:14.568189 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39943d6f-0018-47c3-bd28-13aa4048de53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:54:14.570813 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39943d6f-0018-47c3-bd28-13aa4048de53 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:54:14.583415 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1165/2330] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:54:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:54:22.459864 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=10 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:54:24.578091 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=19 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:56:13.825670 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642064fc-c4cf-4ce8-8358-9c39115ad1a9 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:56:13.838411 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642064fc-c4cf-4ce8-8358-9c39115ad1a9 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:56:13.844753 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fcd043-6d0a-4c6d-aba8-a3b97abe31d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:56:13.845846 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fcd043-6d0a-4c6d-aba8-a3b97abe31d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:56:13.846380 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fcd043-6d0a-4c6d-aba8-a3b97abe31d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:56:13.846515 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fcd043-6d0a-4c6d-aba8-a3b97abe31d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:56:13.848775 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-82fcd043-6d0a-4c6d-aba8-a3b97abe31d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:56:13.953411 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fcd043-6d0a-4c6d-aba8-a3b97abe31d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:56:13.954867 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fcd043-6d0a-4c6d-aba8-a3b97abe31d9 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:56:13.975706 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1166/2331] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:56:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:56:14.934917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39943d6f-0018-47c3-bd28-13aa4048de53 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:56:14.937256 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-39943d6f-0018-47c3-bd28-13aa4048de53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:56:15.013981 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39943d6f-0018-47c3-bd28-13aa4048de53 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:56:15.017694 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3161a368-299d-4232-b14a-988373cc3dab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:56:15.018364 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3161a368-299d-4232-b14a-988373cc3dab service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:56:15.018654 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3161a368-299d-4232-b14a-988373cc3dab service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:56:15.018936 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3161a368-299d-4232-b14a-988373cc3dab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:56:15.021507 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3161a368-299d-4232-b14a-988373cc3dab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:56:15.110799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3161a368-299d-4232-b14a-988373cc3dab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:56:15.112310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3161a368-299d-4232-b14a-988373cc3dab service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:56:15.132782 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1166/2332] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:56:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:56:16.346663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fcd043-6d0a-4c6d-aba8-a3b97abe31d9 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:56:16.357118 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fcd043-6d0a-4c6d-aba8-a3b97abe31d9 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:56:16.360925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e5152-cba6-4031-8a02-3182b3b14c02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:56:16.361699 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e5152-cba6-4031-8a02-3182b3b14c02 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:56:16.361985 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e5152-cba6-4031-8a02-3182b3b14c02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:56:16.362263 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e5152-cba6-4031-8a02-3182b3b14c02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:56:16.365205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4f5e5152-cba6-4031-8a02-3182b3b14c02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:56:16.380419 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5e5152-cba6-4031-8a02-3182b3b14c02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:56:16.381889 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5e5152-cba6-4031-8a02-3182b3b14c02 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:56:16.393612 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1167/2333] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:56:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:56:25.121486 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=64 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:56:26.388556 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=51 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:56:38.048955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3161a368-299d-4232-b14a-988373cc3dab service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:56:38.062856 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3161a368-299d-4232-b14a-988373cc3dab service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:56:38.068277 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5570f1dc-434a-4942-862d-5b7ea74d7d1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:56:38.069567 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570f1dc-434a-4942-862d-5b7ea74d7d1c service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:56:38.070079 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570f1dc-434a-4942-862d-5b7ea74d7d1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:56:38.070798 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570f1dc-434a-4942-862d-5b7ea74d7d1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:56:38.080792 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5570f1dc-434a-4942-862d-5b7ea74d7d1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:56:38.096775 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5570f1dc-434a-4942-862d-5b7ea74d7d1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:56:38.100596 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5570f1dc-434a-4942-862d-5b7ea74d7d1c service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:56:38.118946 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1167/2334] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:56:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:56:48.109555 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:56:59.279493 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585f5bd-b3d0-45df-bb60-01562cbaec24 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:56:59.280491 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585f5bd-b3d0-45df-bb60-01562cbaec24 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:56:59.280673 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585f5bd-b3d0-45df-bb60-01562cbaec24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:56:59.302121 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d585f5bd-b3d0-45df-bb60-01562cbaec24 None None] MFA Rules not processed for user `6ecf34ceee294b53b5be7c1723b42950`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:56:59.303557 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d585f5bd-b3d0-45df-bb60-01562cbaec24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:56:59.311877 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1168/2335] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:56:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:56:59.703329 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5570f1dc-434a-4942-862d-5b7ea74d7d1c service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:56:59.713066 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5570f1dc-434a-4942-862d-5b7ea74d7d1c service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:56:59.716921 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1e668-638f-4849-8ec5-de5a051c07a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:56:59.717587 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1e668-638f-4849-8ec5-de5a051c07a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:56:59.717876 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1e668-638f-4849-8ec5-de5a051c07a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:56:59.718155 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1e668-638f-4849-8ec5-de5a051c07a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:56:59.727958 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1e668-638f-4849-8ec5-de5a051c07a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:56:59.729892 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1e668-638f-4849-8ec5-de5a051c07a8 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:56:59.741674 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1168/2336] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:56:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:57:00.135447 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585f5bd-b3d0-45df-bb60-01562cbaec24 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:57:00.152635 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585f5bd-b3d0-45df-bb60-01562cbaec24 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:57:00.161481 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec3016-5a86-476e-8eed-5ca5be6baa3e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0422d6695bf14925bf0f993cb16e64ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:57:00.162656 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec3016-5a86-476e-8eed-5ca5be6baa3e service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:57:00.164930 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec3016-5a86-476e-8eed-5ca5be6baa3e service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:57:00.164930 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec3016-5a86-476e-8eed-5ca5be6baa3e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:57:00.181492 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec3016-5a86-476e-8eed-5ca5be6baa3e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:57:00.183403 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec3016-5a86-476e-8eed-5ca5be6baa3e service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:57:00.199102 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1169/2337] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:57:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:57:09.735859 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:57:10.190133 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=9 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:57:10.958864 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1e668-638f-4849-8ec5-de5a051c07a8 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:57:10.975704 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1e668-638f-4849-8ec5-de5a051c07a8 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:57:10.981503 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a1eec-5c49-4a10-bff1-e31e7d34eead service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:57:10.982170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a1eec-5c49-4a10-bff1-e31e7d34eead service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:57:10.982532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a1eec-5c49-4a10-bff1-e31e7d34eead service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:57:10.982779 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a1eec-5c49-4a10-bff1-e31e7d34eead service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:57:10.993805 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a1eec-5c49-4a10-bff1-e31e7d34eead service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:57:10.995610 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a1eec-5c49-4a10-bff1-e31e7d34eead service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:57:11.008335 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1169/2338] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:57:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:57:15.544684 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec3016-5a86-476e-8eed-5ca5be6baa3e service placement] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:57:15.560449 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec3016-5a86-476e-8eed-5ca5be6baa3e service placement] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:57:15.566311 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626b081a-0fec-44cc-a655-0a5dcc2b665a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:57:15.566969 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b081a-0fec-44cc-a655-0a5dcc2b665a service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:57:15.567290 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b081a-0fec-44cc-a655-0a5dcc2b665a service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:57:15.567596 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b081a-0fec-44cc-a655-0a5dcc2b665a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:57:15.583249 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626b081a-0fec-44cc-a655-0a5dcc2b665a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:57:15.585123 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626b081a-0fec-44cc-a655-0a5dcc2b665a service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:57:15.597034 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1170/2339] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:57:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:57:21.002185 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:57:25.591950 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:58:14.808339 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a1eec-5c49-4a10-bff1-e31e7d34eead service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:14.810624 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-488a1eec-5c49-4a10-bff1-e31e7d34eead service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:58:14.899696 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a1eec-5c49-4a10-bff1-e31e7d34eead service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:14.903313 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc7137-7418-467c-80e2-caee76348535 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:14.904001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc7137-7418-467c-80e2-caee76348535 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:14.904291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc7137-7418-467c-80e2-caee76348535 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:14.904574 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc7137-7418-467c-80e2-caee76348535 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:14.906524 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e6bc7137-7418-467c-80e2-caee76348535 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:58:15.002553 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc7137-7418-467c-80e2-caee76348535 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:15.004150 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc7137-7418-467c-80e2-caee76348535 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:15.028386 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1170/2340] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:58:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:58:15.078787 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626b081a-0fec-44cc-a655-0a5dcc2b665a service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:15.092179 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626b081a-0fec-44cc-a655-0a5dcc2b665a service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:15.095940 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceaf126-1c9d-442f-b7ba-cad4df905ec8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:15.096594 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceaf126-1c9d-442f-b7ba-cad4df905ec8 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:15.096876 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceaf126-1c9d-442f-b7ba-cad4df905ec8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:15.097151 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceaf126-1c9d-442f-b7ba-cad4df905ec8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:15.100942 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3ceaf126-1c9d-442f-b7ba-cad4df905ec8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:58:15.198702 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceaf126-1c9d-442f-b7ba-cad4df905ec8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:15.200228 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceaf126-1c9d-442f-b7ba-cad4df905ec8 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:15.221440 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1171/2341] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:58:15 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:58:20.705515 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc7137-7418-467c-80e2-caee76348535 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:20.706963 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e6bc7137-7418-467c-80e2-caee76348535 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:58:20.720240 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc7137-7418-467c-80e2-caee76348535 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:20.724565 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde6656-4650-4dad-83ac-2fff97cfdd8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:20.725237 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde6656-4650-4dad-83ac-2fff97cfdd8f service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:20.725530 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde6656-4650-4dad-83ac-2fff97cfdd8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:20.725808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde6656-4650-4dad-83ac-2fff97cfdd8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:20.728672 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ccde6656-4650-4dad-83ac-2fff97cfdd8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:58:20.807818 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:58:20.821942 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde6656-4650-4dad-83ac-2fff97cfdd8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:20.823408 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde6656-4650-4dad-83ac-2fff97cfdd8f service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:20.845695 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1171/2342] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:58:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:58:24.400127 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceaf126-1c9d-442f-b7ba-cad4df905ec8 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:24.401566 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3ceaf126-1c9d-442f-b7ba-cad4df905ec8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:58:24.470761 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde6656-4650-4dad-83ac-2fff97cfdd8f service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:24.474847 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ccde6656-4650-4dad-83ac-2fff97cfdd8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:58:24.490202 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde6656-4650-4dad-83ac-2fff97cfdd8f service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:24.495429 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d2a2e-03c8-4b23-89b6-9dd2e5c27dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:24.501909 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d2a2e-03c8-4b23-89b6-9dd2e5c27dab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:24.501909 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d2a2e-03c8-4b23-89b6-9dd2e5c27dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:24.501909 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d2a2e-03c8-4b23-89b6-9dd2e5c27dab admin admin] PATH_INFO: `/v3/users/0ed3ed6183d34c73ba40d13af037f838` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:24.501909 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07d2a2e-03c8-4b23-89b6-9dd2e5c27dab admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ed3ed6183d34c73ba40d13af037f838)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:24.501909 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07d2a2e-03c8-4b23-89b6-9dd2e5c27dab admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:24.515613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceaf126-1c9d-442f-b7ba-cad4df905ec8 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:24.522262 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e679df-4cdb-45a8-9f54-5f4cecdd5a76 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0422d6695bf14925bf0f993cb16e64ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:24.522934 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e679df-4cdb-45a8-9f54-5f4cecdd5a76 service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:24.523224 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e679df-4cdb-45a8-9f54-5f4cecdd5a76 service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:24.523505 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e679df-4cdb-45a8-9f54-5f4cecdd5a76 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:24.525422 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a1e679df-4cdb-45a8-9f54-5f4cecdd5a76 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:58:24.541053 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e679df-4cdb-45a8-9f54-5f4cecdd5a76 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:24.543204 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e679df-4cdb-45a8-9f54-5f4cecdd5a76 service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:24.555280 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1172/2343] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:58:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:58:24.559742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d07d2a2e-03c8-4b23-89b6-9dd2e5c27dab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ed3ed6183d34c73ba40d13af037f838'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:24.567310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d07d2a2e-03c8-4b23-89b6-9dd2e5c27dab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ed3ed6183d34c73ba40d13af037f838'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:24.576941 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d07d2a2e-03c8-4b23-89b6-9dd2e5c27dab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ed3ed6183d34c73ba40d13af037f838'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:24.577651 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d07d2a2e-03c8-4b23-89b6-9dd2e5c27dab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ed3ed6183d34c73ba40d13af037f838'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:24.578601 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d07d2a2e-03c8-4b23-89b6-9dd2e5c27dab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ed3ed6183d34c73ba40d13af037f838'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:24.602053 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1172/2344] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:58:24 2022] DELETE /identity/v3/users/0ed3ed6183d34c73ba40d13af037f838 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:58:24.816172 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e679df-4cdb-45a8-9f54-5f4cecdd5a76 service placement] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:24.829440 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e679df-4cdb-45a8-9f54-5f4cecdd5a76 service placement] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:24.889394 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:58:24.909467 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c0033a-59d4-4061-b4ed-10945255237b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:24.910456 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c0033a-59d4-4061-b4ed-10945255237b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:24.911249 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c0033a-59d4-4061-b4ed-10945255237b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:24.911737 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c0033a-59d4-4061-b4ed-10945255237b admin admin] PATH_INFO: `/v3/projects/1df108d610134859919e6bf8903f6190` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:24.913053 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c0033a-59d4-4061-b4ed-10945255237b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1df108d610134859919e6bf8903f6190)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:24.915001 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c0033a-59d4-4061-b4ed-10945255237b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:24.969332 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-17c0033a-59d4-4061-b4ed-10945255237b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1df108d610134859919e6bf8903f6190', 'request_id': 'req-17c0033a-59d4-4061-b4ed-10945255237b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:24.978087 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-17c0033a-59d4-4061-b4ed-10945255237b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1df108d610134859919e6bf8903f6190', 'request_id': 'req-17c0033a-59d4-4061-b4ed-10945255237b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:24.984139 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-17c0033a-59d4-4061-b4ed-10945255237b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1df108d610134859919e6bf8903f6190', 'request_id': 'req-17c0033a-59d4-4061-b4ed-10945255237b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:24.996519 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-17c0033a-59d4-4061-b4ed-10945255237b admin admin] The token cache is being invalidate because project 1df108d610134859919e6bf8903f6190 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:58:24.996855 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-17c0033a-59d4-4061-b4ed-10945255237b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:24.997476 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-17c0033a-59d4-4061-b4ed-10945255237b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:24.999134 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1173/2345] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:58:24 2022] DELETE /identity/v3/projects/1df108d610134859919e6bf8903f6190 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:58:25.005834 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae119dc6-04d2-4e1b-804a-a7e8afd69afe None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:25.006127 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae119dc6-04d2-4e1b-804a-a7e8afd69afe None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:25.006407 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae119dc6-04d2-4e1b-804a-a7e8afd69afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:25.069721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ae119dc6-04d2-4e1b-804a-a7e8afd69afe None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:58:25.070856 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ae119dc6-04d2-4e1b-804a-a7e8afd69afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:58:25.137788 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1173/2346] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:58:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:58:25.148537 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b2f8b-b504-43ed-8e8f-8795688fdcfd None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:25.148990 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b2f8b-b504-43ed-8e8f-8795688fdcfd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:25.149742 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b2f8b-b504-43ed-8e8f-8795688fdcfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:25.179677 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b42b2f8b-b504-43ed-8e8f-8795688fdcfd None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:58:25.181033 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b42b2f8b-b504-43ed-8e8f-8795688fdcfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:58:25.193560 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1174/2347] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:58:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:58:25.198077 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae119dc6-04d2-4e1b-804a-a7e8afd69afe None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:25.209566 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae119dc6-04d2-4e1b-804a-a7e8afd69afe None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:25.215094 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30528f7-c32b-4eeb-8344-0801e1a9389d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:25.216246 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30528f7-c32b-4eeb-8344-0801e1a9389d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:25.216407 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30528f7-c32b-4eeb-8344-0801e1a9389d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:25.216778 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30528f7-c32b-4eeb-8344-0801e1a9389d admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:25.219114 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30528f7-c32b-4eeb-8344-0801e1a9389d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:58:25.219114 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30528f7-c32b-4eeb-8344-0801e1a9389d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:25.220117 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30528f7-c32b-4eeb-8344-0801e1a9389d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:25.239001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1174/2348] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:58:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:58:25.242317 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b2f8b-b504-43ed-8e8f-8795688fdcfd None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:25.255887 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b2f8b-b504-43ed-8e8f-8795688fdcfd None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:25.262979 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e001f3-2c2c-404f-9500-6d9e59a7c159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:25.262979 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e001f3-2c2c-404f-9500-6d9e59a7c159 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:25.262979 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e001f3-2c2c-404f-9500-6d9e59a7c159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:25.262979 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e001f3-2c2c-404f-9500-6d9e59a7c159 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:25.263582 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e001f3-2c2c-404f-9500-6d9e59a7c159 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:25.265396 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e001f3-2c2c-404f-9500-6d9e59a7c159 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:25.311714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1175/2349] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:58:25 2022] POST /identity/v3/projects => generated 403 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:58:25.312890 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30528f7-c32b-4eeb-8344-0801e1a9389d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:25.324802 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30528f7-c32b-4eeb-8344-0801e1a9389d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:25.330819 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:58:25.398938 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23cc238-ca3b-4a82-9ecf-982c4134cf1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:25.399684 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23cc238-ca3b-4a82-9ecf-982c4134cf1b admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:25.399969 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23cc238-ca3b-4a82-9ecf-982c4134cf1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:25.400242 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23cc238-ca3b-4a82-9ecf-982c4134cf1b admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:25.400951 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23cc238-ca3b-4a82-9ecf-982c4134cf1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:25.402914 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23cc238-ca3b-4a82-9ecf-982c4134cf1b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:25.428989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1175/2350] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:58:25 2022] POST /identity/v3/users => generated 302 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:58:25.433317 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e001f3-2c2c-404f-9500-6d9e59a7c159 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:25.446315 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e001f3-2c2c-404f-9500-6d9e59a7c159 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:25.451114 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf52460-d631-4aa8-a08b-115bf9fe1dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:25.451835 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf52460-d631-4aa8-a08b-115bf9fe1dec admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:25.452123 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf52460-d631-4aa8-a08b-115bf9fe1dec admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:25.452410 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf52460-d631-4aa8-a08b-115bf9fe1dec admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:25.453178 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf52460-d631-4aa8-a08b-115bf9fe1dec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:25.454808 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf52460-d631-4aa8-a08b-115bf9fe1dec admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:25.465835 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1176/2351] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:58:25.469947 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23cc238-ca3b-4a82-9ecf-982c4134cf1b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:25.480967 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23cc238-ca3b-4a82-9ecf-982c4134cf1b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:25.485971 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872eb89c-4760-4762-bf74-3b72e0bcc822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:25.486641 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872eb89c-4760-4762-bf74-3b72e0bcc822 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:25.486925 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872eb89c-4760-4762-bf74-3b72e0bcc822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:25.487201 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872eb89c-4760-4762-bf74-3b72e0bcc822 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:25.488032 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872eb89c-4760-4762-bf74-3b72e0bcc822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:25.490692 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872eb89c-4760-4762-bf74-3b72e0bcc822 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:25.505167 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1176/2352] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:58:25.511968 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf52460-d631-4aa8-a08b-115bf9fe1dec admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:25.527272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf52460-d631-4aa8-a08b-115bf9fe1dec admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:25.532032 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3ab0b-52d5-4d70-bded-95a263b37496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:25.532910 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3ab0b-52d5-4d70-bded-95a263b37496 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:25.533192 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3ab0b-52d5-4d70-bded-95a263b37496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:25.533466 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3ab0b-52d5-4d70-bded-95a263b37496 admin admin] PATH_INFO: `/v3/projects/f902d49c2be84f20938c706e683b2795/users/26994fb2369e4ab28d87076bd74c9af9/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:25.549724 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca3ab0b-52d5-4d70-bded-95a263b37496 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f902d49c2be84f20938c706e683b2795, user_id=26994fb2369e4ab28d87076bd74c9af9, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:25.553519 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca3ab0b-52d5-4d70-bded-95a263b37496 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:25.568374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1177/2353] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:58:25 2022] PUT /identity/v3/projects/f902d49c2be84f20938c706e683b2795/users/26994fb2369e4ab28d87076bd74c9af9/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:58:25.573970 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43b9a6-852f-4d61-b3a1-b8e6de5c423c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:25.574264 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43b9a6-852f-4d61-b3a1-b8e6de5c423c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:25.574541 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43b9a6-852f-4d61-b3a1-b8e6de5c423c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:25.597339 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ec43b9a6-852f-4d61-b3a1-b8e6de5c423c None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:58:25.599439 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ec43b9a6-852f-4d61-b3a1-b8e6de5c423c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:58:25.664445 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1177/2354] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:58:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:58:25.674441 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3ab0b-52d5-4d70-bded-95a263b37496 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:25.676589 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3ca3ab0b-52d5-4d70-bded-95a263b37496 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:58:25.740433 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3ab0b-52d5-4d70-bded-95a263b37496 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:25.744178 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3ec12-4d48-47ed-aa78-8f136c7e4692 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:25.744830 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3ec12-4d48-47ed-aa78-8f136c7e4692 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:25.745114 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3ec12-4d48-47ed-aa78-8f136c7e4692 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:25.745392 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3ec12-4d48-47ed-aa78-8f136c7e4692 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:25.755851 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3ec12-4d48-47ed-aa78-8f136c7e4692 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:25.757422 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3ec12-4d48-47ed-aa78-8f136c7e4692 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:25.768000 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1178/2355] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:58:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:58:26.780069 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25213a-b10c-465b-90aa-5c9d4c8cb1d9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:26.780704 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25213a-b10c-465b-90aa-5c9d4c8cb1d9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:26.780704 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25213a-b10c-465b-90aa-5c9d4c8cb1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:26.814948 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-cd25213a-b10c-465b-90aa-5c9d4c8cb1d9 None None] MFA Rules not processed for user `26994fb2369e4ab28d87076bd74c9af9`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:58:26.816253 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cd25213a-b10c-465b-90aa-5c9d4c8cb1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:58:26.890760 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1178/2356] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:58:26 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:58:26.896077 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3ec12-4d48-47ed-aa78-8f136c7e4692 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:26.908530 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3ec12-4d48-47ed-aa78-8f136c7e4692 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:26.914993 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5588c504-da68-48c7-a018-cb25fd3ab948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:26.915624 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588c504-da68-48c7-a018-cb25fd3ab948 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:26.915913 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588c504-da68-48c7-a018-cb25fd3ab948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:26.916194 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588c504-da68-48c7-a018-cb25fd3ab948 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:26.916897 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5588c504-da68-48c7-a018-cb25fd3ab948 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:26.918896 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5588c504-da68-48c7-a018-cb25fd3ab948 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:26.924899 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:58:26.956588 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1179/2357] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:58:26 2022] POST /identity/v3/projects => generated 403 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:58:26.957948 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd25213a-b10c-465b-90aa-5c9d4c8cb1d9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:26.969266 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd25213a-b10c-465b-90aa-5c9d4c8cb1d9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:26.983953 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:58:27.031987 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a655c5-8fe0-47f6-bc44-124b4d36962c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:27.032683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a655c5-8fe0-47f6-bc44-124b4d36962c admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:27.032976 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a655c5-8fe0-47f6-bc44-124b4d36962c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:27.033259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a655c5-8fe0-47f6-bc44-124b4d36962c admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:27.033965 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a655c5-8fe0-47f6-bc44-124b4d36962c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:27.037398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a655c5-8fe0-47f6-bc44-124b4d36962c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:27.059472 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1179/2358] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:58:26 2022] POST /identity/v3/users => generated 308 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:58:27.063532 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5588c504-da68-48c7-a018-cb25fd3ab948 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:27.074479 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5588c504-da68-48c7-a018-cb25fd3ab948 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:27.079238 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40686b5d-33eb-41cb-8e94-b5b1faee59bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:27.079939 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40686b5d-33eb-41cb-8e94-b5b1faee59bf admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:27.080228 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40686b5d-33eb-41cb-8e94-b5b1faee59bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:27.082628 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40686b5d-33eb-41cb-8e94-b5b1faee59bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:27.082628 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40686b5d-33eb-41cb-8e94-b5b1faee59bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:27.083454 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40686b5d-33eb-41cb-8e94-b5b1faee59bf admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:27.099755 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1180/2359] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:58:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:58:27.104687 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a655c5-8fe0-47f6-bc44-124b4d36962c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:27.118721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a655c5-8fe0-47f6-bc44-124b4d36962c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:27.123845 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4022a47-3021-40ec-8caa-fa568facf77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:27.124828 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4022a47-3021-40ec-8caa-fa568facf77c admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:27.125117 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4022a47-3021-40ec-8caa-fa568facf77c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:27.125401 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4022a47-3021-40ec-8caa-fa568facf77c admin admin] PATH_INFO: `/v3/projects/1f76093434314fb98ad7ec89a9347e86/users/44467554331c458094ce7490ad1ec01c/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:27.142909 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4022a47-3021-40ec-8caa-fa568facf77c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f76093434314fb98ad7ec89a9347e86, user_id=44467554331c458094ce7490ad1ec01c, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:27.146048 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4022a47-3021-40ec-8caa-fa568facf77c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:27.162064 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1180/2360] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:58:27 2022] PUT /identity/v3/projects/1f76093434314fb98ad7ec89a9347e86/users/44467554331c458094ce7490ad1ec01c/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:58:27.170971 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40686b5d-33eb-41cb-8e94-b5b1faee59bf admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:27.184508 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40686b5d-33eb-41cb-8e94-b5b1faee59bf admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:27.257853 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed2c48c-f4b0-4f81-842b-9d3e166ce8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:27.258446 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2c48c-f4b0-4f81-842b-9d3e166ce8ff admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:27.258735 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2c48c-f4b0-4f81-842b-9d3e166ce8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:27.259015 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2c48c-f4b0-4f81-842b-9d3e166ce8ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:27.259815 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed2c48c-f4b0-4f81-842b-9d3e166ce8ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:27.261570 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed2c48c-f4b0-4f81-842b-9d3e166ce8ff admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:27.274365 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1181/2361] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:58:27 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:58:27.278715 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4022a47-3021-40ec-8caa-fa568facf77c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:27.295027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4022a47-3021-40ec-8caa-fa568facf77c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:27.295653 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf3c8e0-0e41-4208-831d-0f3fdace0890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:27.296587 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3c8e0-0e41-4208-831d-0f3fdace0890 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:27.296900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3c8e0-0e41-4208-831d-0f3fdace0890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:27.297179 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3c8e0-0e41-4208-831d-0f3fdace0890 admin admin] PATH_INFO: `/v3/projects/1f76093434314fb98ad7ec89a9347e86/users/44467554331c458094ce7490ad1ec01c/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:27.299222 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf3c8e0-0e41-4208-831d-0f3fdace0890 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f76093434314fb98ad7ec89a9347e86, user_id=44467554331c458094ce7490ad1ec01c, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:27.302959 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf3c8e0-0e41-4208-831d-0f3fdace0890 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:27.314156 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1181/2362] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:58:27 2022] PUT /identity/v3/projects/1f76093434314fb98ad7ec89a9347e86/users/44467554331c458094ce7490ad1ec01c/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:58:28.089685 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ce6d2-24be-4df4-8f11-b6810a90f89f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:28.090293 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ce6d2-24be-4df4-8f11-b6810a90f89f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:28.090445 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ce6d2-24be-4df4-8f11-b6810a90f89f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:28.130361 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-fd5ce6d2-24be-4df4-8f11-b6810a90f89f None None] MFA Rules not processed for user `44467554331c458094ce7490ad1ec01c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:58:28.130361 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fd5ce6d2-24be-4df4-8f11-b6810a90f89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:58:28.221902 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1182/2363] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:58:28 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:58:29.462389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf3c8e0-0e41-4208-831d-0f3fdace0890 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:29.465851 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0cf3c8e0-0e41-4208-831d-0f3fdace0890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:58:29.540267 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ce6d2-24be-4df4-8f11-b6810a90f89f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:29.584283 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf3c8e0-0e41-4208-831d-0f3fdace0890 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:29.589792 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3fed9-59c4-435b-b574-4ac393dc66c3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0422d6695bf14925bf0f993cb16e64ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:29.590466 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3fed9-59c4-435b-b574-4ac393dc66c3 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:29.590749 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3fed9-59c4-435b-b574-4ac393dc66c3 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:29.591028 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3fed9-59c4-435b-b574-4ac393dc66c3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:29.592952 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1ed3fed9-59c4-435b-b574-4ac393dc66c3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:58:29.614604 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed3fed9-59c4-435b-b574-4ac393dc66c3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:29.617377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed3fed9-59c4-435b-b574-4ac393dc66c3 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:29.628874 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:58:29.632022 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1182/2364] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:58:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:58:29.638051 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ce6d2-24be-4df4-8f11-b6810a90f89f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:29.649214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751f738-2a5f-42c8-80a4-1f504f32980c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:29.650637 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751f738-2a5f-42c8-80a4-1f504f32980c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:29.651167 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751f738-2a5f-42c8-80a4-1f504f32980c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:29.651678 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751f738-2a5f-42c8-80a4-1f504f32980c admin admin] PATH_INFO: `/v3/users/26994fb2369e4ab28d87076bd74c9af9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:29.653954 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4751f738-2a5f-42c8-80a4-1f504f32980c admin admin] RBAC: Authorizing `identity:delete_user(user_id=26994fb2369e4ab28d87076bd74c9af9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:29.656631 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4751f738-2a5f-42c8-80a4-1f504f32980c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:29.716631 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4751f738-2a5f-42c8-80a4-1f504f32980c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26994fb2369e4ab28d87076bd74c9af9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:29.727810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4751f738-2a5f-42c8-80a4-1f504f32980c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26994fb2369e4ab28d87076bd74c9af9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:29.741260 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4751f738-2a5f-42c8-80a4-1f504f32980c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26994fb2369e4ab28d87076bd74c9af9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:29.742051 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4751f738-2a5f-42c8-80a4-1f504f32980c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26994fb2369e4ab28d87076bd74c9af9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:29.762306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4751f738-2a5f-42c8-80a4-1f504f32980c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26994fb2369e4ab28d87076bd74c9af9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:29.773087 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1183/2365] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:58:29 2022] DELETE /identity/v3/users/26994fb2369e4ab28d87076bd74c9af9 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:58:30.002409 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3fed9-59c4-435b-b574-4ac393dc66c3 service placement] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:30.014812 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3fed9-59c4-435b-b574-4ac393dc66c3 service placement] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:30.093545 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e7b502-d95a-4d20-8f00-2f34b0916215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:30.094582 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7b502-d95a-4d20-8f00-2f34b0916215 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:30.095064 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7b502-d95a-4d20-8f00-2f34b0916215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:30.095573 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7b502-d95a-4d20-8f00-2f34b0916215 admin admin] PATH_INFO: `/v3/projects/f902d49c2be84f20938c706e683b2795` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:30.097687 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e7b502-d95a-4d20-8f00-2f34b0916215 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f902d49c2be84f20938c706e683b2795)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:30.100819 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e7b502-d95a-4d20-8f00-2f34b0916215 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:30.102701 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:58:30.157536 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-81e7b502-d95a-4d20-8f00-2f34b0916215 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f902d49c2be84f20938c706e683b2795', 'request_id': 'req-81e7b502-d95a-4d20-8f00-2f34b0916215'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:30.165334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-81e7b502-d95a-4d20-8f00-2f34b0916215 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f902d49c2be84f20938c706e683b2795', 'request_id': 'req-81e7b502-d95a-4d20-8f00-2f34b0916215'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:30.173068 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-81e7b502-d95a-4d20-8f00-2f34b0916215 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f902d49c2be84f20938c706e683b2795', 'request_id': 'req-81e7b502-d95a-4d20-8f00-2f34b0916215'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:30.187805 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-81e7b502-d95a-4d20-8f00-2f34b0916215 admin admin] The token cache is being invalidate because project f902d49c2be84f20938c706e683b2795 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:58:30.188099 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-81e7b502-d95a-4d20-8f00-2f34b0916215 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:30.188770 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-81e7b502-d95a-4d20-8f00-2f34b0916215 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:30.190351 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1183/2366] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:58:30 2022] DELETE /identity/v3/projects/f902d49c2be84f20938c706e683b2795 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:58:30.195960 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751f738-2a5f-42c8-80a4-1f504f32980c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:30.196198 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4751f738-2a5f-42c8-80a4-1f504f32980c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:58:30.210400 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751f738-2a5f-42c8-80a4-1f504f32980c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:30.288566 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9320536-3229-4a62-b021-a48dfef3e2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:30.290008 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9320536-3229-4a62-b021-a48dfef3e2b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:30.290396 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9320536-3229-4a62-b021-a48dfef3e2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:30.290714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9320536-3229-4a62-b021-a48dfef3e2b5 admin admin] PATH_INFO: `/v3/users/44467554331c458094ce7490ad1ec01c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:30.292091 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9320536-3229-4a62-b021-a48dfef3e2b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44467554331c458094ce7490ad1ec01c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:30.294089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9320536-3229-4a62-b021-a48dfef3e2b5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:30.355708 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c9320536-3229-4a62-b021-a48dfef3e2b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44467554331c458094ce7490ad1ec01c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:30.367916 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c9320536-3229-4a62-b021-a48dfef3e2b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44467554331c458094ce7490ad1ec01c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:30.376190 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c9320536-3229-4a62-b021-a48dfef3e2b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44467554331c458094ce7490ad1ec01c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:30.376892 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c9320536-3229-4a62-b021-a48dfef3e2b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44467554331c458094ce7490ad1ec01c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:30.389118 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:58:30.390348 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c9320536-3229-4a62-b021-a48dfef3e2b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44467554331c458094ce7490ad1ec01c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:30.400244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1184/2367] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:58:30 2022] DELETE /identity/v3/users/44467554331c458094ce7490ad1ec01c => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:58:30.631975 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e7b502-d95a-4d20-8f00-2f34b0916215 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:30.634004 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-81e7b502-d95a-4d20-8f00-2f34b0916215 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:58:30.654633 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e7b502-d95a-4d20-8f00-2f34b0916215 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:30.734738 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e381c486-8eb6-4c07-aba8-677ff8d1cd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:30.735686 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e381c486-8eb6-4c07-aba8-677ff8d1cd36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:30.735962 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e381c486-8eb6-4c07-aba8-677ff8d1cd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:30.736240 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e381c486-8eb6-4c07-aba8-677ff8d1cd36 admin admin] PATH_INFO: `/v3/projects/1f76093434314fb98ad7ec89a9347e86` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:30.737564 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e381c486-8eb6-4c07-aba8-677ff8d1cd36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f76093434314fb98ad7ec89a9347e86)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:30.739784 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e381c486-8eb6-4c07-aba8-677ff8d1cd36 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:30.796452 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e381c486-8eb6-4c07-aba8-677ff8d1cd36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f76093434314fb98ad7ec89a9347e86', 'request_id': 'req-e381c486-8eb6-4c07-aba8-677ff8d1cd36'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:30.804450 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e381c486-8eb6-4c07-aba8-677ff8d1cd36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f76093434314fb98ad7ec89a9347e86', 'request_id': 'req-e381c486-8eb6-4c07-aba8-677ff8d1cd36'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:30.811050 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e381c486-8eb6-4c07-aba8-677ff8d1cd36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f76093434314fb98ad7ec89a9347e86', 'request_id': 'req-e381c486-8eb6-4c07-aba8-677ff8d1cd36'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:30.827916 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e381c486-8eb6-4c07-aba8-677ff8d1cd36 admin admin] The token cache is being invalidate because project 1f76093434314fb98ad7ec89a9347e86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:58:30.828190 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e381c486-8eb6-4c07-aba8-677ff8d1cd36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:30.828926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e381c486-8eb6-4c07-aba8-677ff8d1cd36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:30.830673 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1184/2368] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:58:30 2022] DELETE /identity/v3/projects/1f76093434314fb98ad7ec89a9347e86 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:58:30.837741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135080a-4e5a-4694-a360-d8421e05407e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:30.838303 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135080a-4e5a-4694-a360-d8421e05407e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:30.838303 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135080a-4e5a-4694-a360-d8421e05407e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:30.865450 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8135080a-4e5a-4694-a360-d8421e05407e None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:58:30.866687 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8135080a-4e5a-4694-a360-d8421e05407e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:58:30.937549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1185/2369] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:58:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:58:30.946464 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac798d-7a4c-4599-bbfc-7de3e3cf56bc None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:30.946788 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac798d-7a4c-4599-bbfc-7de3e3cf56bc None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:30.947077 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac798d-7a4c-4599-bbfc-7de3e3cf56bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:30.968573 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2fac798d-7a4c-4599-bbfc-7de3e3cf56bc None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:58:30.969804 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2fac798d-7a4c-4599-bbfc-7de3e3cf56bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:58:30.980304 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1185/2370] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:58:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:58:30.984731 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8135080a-4e5a-4694-a360-d8421e05407e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:30.995159 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8135080a-4e5a-4694-a360-d8421e05407e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:31.000436 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a051c85a-6cce-498c-8ae4-60a4d743dec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:31.001116 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051c85a-6cce-498c-8ae4-60a4d743dec9 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:31.001402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051c85a-6cce-498c-8ae4-60a4d743dec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:31.001677 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051c85a-6cce-498c-8ae4-60a4d743dec9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:31.002416 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a051c85a-6cce-498c-8ae4-60a4d743dec9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:58:31.002729 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a051c85a-6cce-498c-8ae4-60a4d743dec9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:31.004418 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a051c85a-6cce-498c-8ae4-60a4d743dec9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:31.024023 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1186/2371] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:58:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:58:31.028660 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac798d-7a4c-4599-bbfc-7de3e3cf56bc None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:31.040044 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac798d-7a4c-4599-bbfc-7de3e3cf56bc None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:31.047042 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94bbf8f-f8df-4240-adef-175a5c8891a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:31.047761 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94bbf8f-f8df-4240-adef-175a5c8891a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:31.048050 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94bbf8f-f8df-4240-adef-175a5c8891a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:31.048330 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94bbf8f-f8df-4240-adef-175a5c8891a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:31.049046 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94bbf8f-f8df-4240-adef-175a5c8891a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:31.051154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94bbf8f-f8df-4240-adef-175a5c8891a4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:31.063589 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:58:31.083828 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1186/2372] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:58:31 2022] POST /identity/v3/projects => generated 387 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:58:31.088523 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a051c85a-6cce-498c-8ae4-60a4d743dec9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:31.100333 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a051c85a-6cce-498c-8ae4-60a4d743dec9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:31.186349 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9bffb-a4fa-4960-a3ed-5e2a51ec370a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:31.187052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9bffb-a4fa-4960-a3ed-5e2a51ec370a admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:31.187342 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9bffb-a4fa-4960-a3ed-5e2a51ec370a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:31.187662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9bffb-a4fa-4960-a3ed-5e2a51ec370a admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:31.188377 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed9bffb-a4fa-4960-a3ed-5e2a51ec370a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:31.190387 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed9bffb-a4fa-4960-a3ed-5e2a51ec370a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:31.215958 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1187/2373] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:58:31 2022] POST /identity/v3/users => generated 294 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:58:31.220207 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94bbf8f-f8df-4240-adef-175a5c8891a4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:31.235662 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94bbf8f-f8df-4240-adef-175a5c8891a4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:31.243312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676b5785-8279-4ce3-a215-2586a6bc52c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:31.244058 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b5785-8279-4ce3-a215-2586a6bc52c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:31.244349 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b5785-8279-4ce3-a215-2586a6bc52c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:31.244633 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b5785-8279-4ce3-a215-2586a6bc52c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:31.245936 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676b5785-8279-4ce3-a215-2586a6bc52c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:31.247669 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676b5785-8279-4ce3-a215-2586a6bc52c6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:31.260573 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1187/2374] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:58:31.265362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9bffb-a4fa-4960-a3ed-5e2a51ec370a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:31.275914 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9bffb-a4fa-4960-a3ed-5e2a51ec370a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:31.281009 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f342ebd2-b794-4c7b-952a-aef422540185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:31.281712 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342ebd2-b794-4c7b-952a-aef422540185 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:31.281995 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342ebd2-b794-4c7b-952a-aef422540185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:31.282302 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342ebd2-b794-4c7b-952a-aef422540185 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:31.283046 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f342ebd2-b794-4c7b-952a-aef422540185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:31.284712 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f342ebd2-b794-4c7b-952a-aef422540185 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:31.297017 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1188/2375] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 19:58:31.302397 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676b5785-8279-4ce3-a215-2586a6bc52c6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:31.313078 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676b5785-8279-4ce3-a215-2586a6bc52c6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:31.318305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a4e02a-fe9d-4b9b-9ccf-dc54f99d0df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:31.319138 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a4e02a-fe9d-4b9b-9ccf-dc54f99d0df7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:31.319423 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a4e02a-fe9d-4b9b-9ccf-dc54f99d0df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:31.319732 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a4e02a-fe9d-4b9b-9ccf-dc54f99d0df7 admin admin] PATH_INFO: `/v3/projects/ee5b662eddda4b07a7c0c008f115fe3c/users/968bdfbb242a4c12999b4dd9ce229299/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:31.337487 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a4e02a-fe9d-4b9b-9ccf-dc54f99d0df7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee5b662eddda4b07a7c0c008f115fe3c, user_id=968bdfbb242a4c12999b4dd9ce229299, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:31.340234 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a4e02a-fe9d-4b9b-9ccf-dc54f99d0df7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:31.358196 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1188/2376] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:58:31 2022] PUT /identity/v3/projects/ee5b662eddda4b07a7c0c008f115fe3c/users/968bdfbb242a4c12999b4dd9ce229299/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:58:31.372018 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfa667-afb5-4746-b739-43eab5db196b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:31.372294 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfa667-afb5-4746-b739-43eab5db196b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:31.372646 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfa667-afb5-4746-b739-43eab5db196b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:31.409706 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3abfa667-afb5-4746-b739-43eab5db196b None None] MFA Rules not processed for user `968bdfbb242a4c12999b4dd9ce229299`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:58:31.413321 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3abfa667-afb5-4746-b739-43eab5db196b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:58:31.494052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:58:31.517630 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1189/2377] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:58:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:58:31.527616 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a4e02a-fe9d-4b9b-9ccf-dc54f99d0df7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:31.606787 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a4e02a-fe9d-4b9b-9ccf-dc54f99d0df7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:31.612676 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e34b48-42d2-44af-973e-36ffbd50dabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:31.613440 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e34b48-42d2-44af-973e-36ffbd50dabf admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:31.613872 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e34b48-42d2-44af-973e-36ffbd50dabf admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:31.614155 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e34b48-42d2-44af-973e-36ffbd50dabf admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:31.615130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e34b48-42d2-44af-973e-36ffbd50dabf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:31.617398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e34b48-42d2-44af-973e-36ffbd50dabf admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:31.648241 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1189/2378] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:58:31 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:58:31.652594 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abfa667-afb5-4746-b739-43eab5db196b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:31.665615 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abfa667-afb5-4746-b739-43eab5db196b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:31.731006 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf036683-5c04-496b-ab6c-dc2cc756c05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:31.731735 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf036683-5c04-496b-ab6c-dc2cc756c05b admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:31.732025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf036683-5c04-496b-ab6c-dc2cc756c05b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:31.732309 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf036683-5c04-496b-ab6c-dc2cc756c05b admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:31.733021 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf036683-5c04-496b-ab6c-dc2cc756c05b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:31.735011 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf036683-5c04-496b-ab6c-dc2cc756c05b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:31.759815 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1190/2379] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:58:31 2022] POST /identity/v3/users => generated 299 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 19:58:31.765109 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e34b48-42d2-44af-973e-36ffbd50dabf admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:31.777356 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e34b48-42d2-44af-973e-36ffbd50dabf admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:31.782377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb3038-30c3-439c-8b97-a42bc346df33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:31.783067 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb3038-30c3-439c-8b97-a42bc346df33 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:31.783360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb3038-30c3-439c-8b97-a42bc346df33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:31.783680 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb3038-30c3-439c-8b97-a42bc346df33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:31.784432 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb3038-30c3-439c-8b97-a42bc346df33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:31.786979 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb3038-30c3-439c-8b97-a42bc346df33 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:31.800836 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1190/2380] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:58:31.805980 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf036683-5c04-496b-ab6c-dc2cc756c05b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:31.820704 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf036683-5c04-496b-ab6c-dc2cc756c05b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:31.825836 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0b236-1bca-47bd-96f3-5351ae945757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:31.826748 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0b236-1bca-47bd-96f3-5351ae945757 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:31.827029 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0b236-1bca-47bd-96f3-5351ae945757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:31.827308 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0b236-1bca-47bd-96f3-5351ae945757 admin admin] PATH_INFO: `/v3/projects/0d780817942d4dafb886f6534534262a/users/f87a72599e4c4ee1bec66ba1235cfa7c/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:31.846327 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e0b236-1bca-47bd-96f3-5351ae945757 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d780817942d4dafb886f6534534262a, user_id=f87a72599e4c4ee1bec66ba1235cfa7c, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:31.852784 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e0b236-1bca-47bd-96f3-5351ae945757 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:31.875846 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1191/2381] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:58:31 2022] PUT /identity/v3/projects/0d780817942d4dafb886f6534534262a/users/f87a72599e4c4ee1bec66ba1235cfa7c/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:58:31.880249 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb3038-30c3-439c-8b97-a42bc346df33 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:31.897839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb3038-30c3-439c-8b97-a42bc346df33 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:31.971378 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b62c79-96b3-4a1a-ae7c-20a4de9fd4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:31.972094 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b62c79-96b3-4a1a-ae7c-20a4de9fd4f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:31.972380 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b62c79-96b3-4a1a-ae7c-20a4de9fd4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:31.972743 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b62c79-96b3-4a1a-ae7c-20a4de9fd4f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:31.974257 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b62c79-96b3-4a1a-ae7c-20a4de9fd4f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:31.975946 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b62c79-96b3-4a1a-ae7c-20a4de9fd4f6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:31.986827 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1191/2382] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:58:31.994364 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0b236-1bca-47bd-96f3-5351ae945757 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:32.008219 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0b236-1bca-47bd-96f3-5351ae945757 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:32.014140 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87dc93-4c02-4c9e-ab4a-e269910e5c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:32.016383 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87dc93-4c02-4c9e-ab4a-e269910e5c24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:32.016383 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87dc93-4c02-4c9e-ab4a-e269910e5c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:32.016383 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87dc93-4c02-4c9e-ab4a-e269910e5c24 admin admin] PATH_INFO: `/v3/projects/0d780817942d4dafb886f6534534262a/users/f87a72599e4c4ee1bec66ba1235cfa7c/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:32.018749 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d87dc93-4c02-4c9e-ab4a-e269910e5c24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d780817942d4dafb886f6534534262a, user_id=f87a72599e4c4ee1bec66ba1235cfa7c, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:32.023562 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d87dc93-4c02-4c9e-ab4a-e269910e5c24 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:32.034834 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1192/2383] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:58:31 2022] PUT /identity/v3/projects/0d780817942d4dafb886f6534534262a/users/f87a72599e4c4ee1bec66ba1235cfa7c/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:58:32.045841 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eccf592-ec02-434a-86d7-bd1f496d1d18 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:32.046094 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eccf592-ec02-434a-86d7-bd1f496d1d18 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:32.046390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eccf592-ec02-434a-86d7-bd1f496d1d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:32.088339 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9eccf592-ec02-434a-86d7-bd1f496d1d18 None None] MFA Rules not processed for user `f87a72599e4c4ee1bec66ba1235cfa7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:58:32.089838 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9eccf592-ec02-434a-86d7-bd1f496d1d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:58:32.105834 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:58:32.189348 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1192/2384] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:58:32 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:58:42.029767 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=48,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:58:42.184360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=22 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:58:46.360947 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87dc93-4c02-4c9e-ab4a-e269910e5c24 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:46.363076 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7d87dc93-4c02-4c9e-ab4a-e269910e5c24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:58:46.458313 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87dc93-4c02-4c9e-ab4a-e269910e5c24 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:46.463749 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c816bba-a69e-413c-a599-a7d65e34d026 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0422d6695bf14925bf0f993cb16e64ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:46.464468 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c816bba-a69e-413c-a599-a7d65e34d026 service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:46.464926 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c816bba-a69e-413c-a599-a7d65e34d026 service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:46.465108 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c816bba-a69e-413c-a599-a7d65e34d026 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:46.467075 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9c816bba-a69e-413c-a599-a7d65e34d026 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:58:46.485119 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c816bba-a69e-413c-a599-a7d65e34d026 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:46.486955 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c816bba-a69e-413c-a599-a7d65e34d026 service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:46.498885 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1193/2385] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:58:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 19:58:52.919623 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eccf592-ec02-434a-86d7-bd1f496d1d18 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:52.921279 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9eccf592-ec02-434a-86d7-bd1f496d1d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:58:53.009404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eccf592-ec02-434a-86d7-bd1f496d1d18 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:53.013628 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e47d4-065a-4cff-9c8f-fba90c66ba80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:53.014298 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e47d4-065a-4cff-9c8f-fba90c66ba80 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:53.014581 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e47d4-065a-4cff-9c8f-fba90c66ba80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:53.015023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e47d4-065a-4cff-9c8f-fba90c66ba80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:53.027728 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619e47d4-065a-4cff-9c8f-fba90c66ba80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:53.030576 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619e47d4-065a-4cff-9c8f-fba90c66ba80 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:53.046052 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1193/2386] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:58:52 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:58:53.080457 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c816bba-a69e-413c-a599-a7d65e34d026 service placement] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:53.186536 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c816bba-a69e-413c-a599-a7d65e34d026 service placement] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:53.193228 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f51ef-6a39-4cd5-9c84-77fdc69d7e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:53.194022 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f51ef-6a39-4cd5-9c84-77fdc69d7e82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:53.194312 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f51ef-6a39-4cd5-9c84-77fdc69d7e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:53.194595 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f51ef-6a39-4cd5-9c84-77fdc69d7e82 admin admin] PATH_INFO: `/v3/users/968bdfbb242a4c12999b4dd9ce229299` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:53.196060 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f51ef-6a39-4cd5-9c84-77fdc69d7e82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=968bdfbb242a4c12999b4dd9ce229299)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:53.198058 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f51ef-6a39-4cd5-9c84-77fdc69d7e82 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:53.270864 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3f3f51ef-6a39-4cd5-9c84-77fdc69d7e82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '968bdfbb242a4c12999b4dd9ce229299'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:53.280804 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3f3f51ef-6a39-4cd5-9c84-77fdc69d7e82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '968bdfbb242a4c12999b4dd9ce229299'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:53.290865 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3f3f51ef-6a39-4cd5-9c84-77fdc69d7e82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '968bdfbb242a4c12999b4dd9ce229299'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:53.292300 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3f3f51ef-6a39-4cd5-9c84-77fdc69d7e82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '968bdfbb242a4c12999b4dd9ce229299'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:53.313293 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3f3f51ef-6a39-4cd5-9c84-77fdc69d7e82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '968bdfbb242a4c12999b4dd9ce229299'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:53.323945 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1194/2387] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:58:53 2022] DELETE /identity/v3/users/968bdfbb242a4c12999b4dd9ce229299 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:58:53.330392 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1d8f2-b8e8-460d-96ba-32dc3fbe38ab None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:53.330864 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1d8f2-b8e8-460d-96ba-32dc3fbe38ab None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:53.331314 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1d8f2-b8e8-460d-96ba-32dc3fbe38ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:53.354871 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a7f1d8f2-b8e8-460d-96ba-32dc3fbe38ab None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:58:53.356233 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a7f1d8f2-b8e8-460d-96ba-32dc3fbe38ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:58:53.438234 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1194/2388] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:58:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:58:53.451489 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f51ef-6a39-4cd5-9c84-77fdc69d7e82 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:53.452876 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3f3f51ef-6a39-4cd5-9c84-77fdc69d7e82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:58:53.466247 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:58:53.532070 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f51ef-6a39-4cd5-9c84-77fdc69d7e82 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:53.537440 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53139bdd-c6be-480f-b469-3834410fa047 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:53.538411 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53139bdd-c6be-480f-b469-3834410fa047 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:53.538857 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53139bdd-c6be-480f-b469-3834410fa047 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:53.539307 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53139bdd-c6be-480f-b469-3834410fa047 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:53.550623 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53139bdd-c6be-480f-b469-3834410fa047 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:53.552227 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53139bdd-c6be-480f-b469-3834410fa047 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:53.564618 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1195/2389] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:58:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:58:53.900643 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1d8f2-b8e8-460d-96ba-32dc3fbe38ab None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:53.901966 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a7f1d8f2-b8e8-460d-96ba-32dc3fbe38ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:58:53.918419 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1d8f2-b8e8-460d-96ba-32dc3fbe38ab None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:53.928540 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3eb3f5-5847-408d-8b5f-bb86806df605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:53.930606 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3eb3f5-5847-408d-8b5f-bb86806df605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:53.930789 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3eb3f5-5847-408d-8b5f-bb86806df605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:53.931269 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3eb3f5-5847-408d-8b5f-bb86806df605 admin admin] PATH_INFO: `/v3/projects/ee5b662eddda4b07a7c0c008f115fe3c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:53.932685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3eb3f5-5847-408d-8b5f-bb86806df605 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee5b662eddda4b07a7c0c008f115fe3c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:53.934822 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3eb3f5-5847-408d-8b5f-bb86806df605 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:53.998941 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ac3eb3f5-5847-408d-8b5f-bb86806df605 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee5b662eddda4b07a7c0c008f115fe3c', 'request_id': 'req-ac3eb3f5-5847-408d-8b5f-bb86806df605'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:54.011081 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ac3eb3f5-5847-408d-8b5f-bb86806df605 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee5b662eddda4b07a7c0c008f115fe3c', 'request_id': 'req-ac3eb3f5-5847-408d-8b5f-bb86806df605'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:54.018877 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ac3eb3f5-5847-408d-8b5f-bb86806df605 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee5b662eddda4b07a7c0c008f115fe3c', 'request_id': 'req-ac3eb3f5-5847-408d-8b5f-bb86806df605'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:54.036062 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ac3eb3f5-5847-408d-8b5f-bb86806df605 admin admin] The token cache is being invalidate because project ee5b662eddda4b07a7c0c008f115fe3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:58:54.036620 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ac3eb3f5-5847-408d-8b5f-bb86806df605 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:54.037574 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ac3eb3f5-5847-408d-8b5f-bb86806df605 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:54.039616 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1195/2390] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:58:53 2022] DELETE /identity/v3/projects/ee5b662eddda4b07a7c0c008f115fe3c => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:58:54.043931 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53139bdd-c6be-480f-b469-3834410fa047 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:54.055849 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53139bdd-c6be-480f-b469-3834410fa047 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:54.141204 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade89b68-f362-4ec3-a51a-40a60b6dc21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:54.142196 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade89b68-f362-4ec3-a51a-40a60b6dc21d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:54.142562 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade89b68-f362-4ec3-a51a-40a60b6dc21d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:54.142831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade89b68-f362-4ec3-a51a-40a60b6dc21d admin admin] PATH_INFO: `/v3/users/f87a72599e4c4ee1bec66ba1235cfa7c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:54.144264 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade89b68-f362-4ec3-a51a-40a60b6dc21d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f87a72599e4c4ee1bec66ba1235cfa7c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:54.146446 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade89b68-f362-4ec3-a51a-40a60b6dc21d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:54.211604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ade89b68-f362-4ec3-a51a-40a60b6dc21d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f87a72599e4c4ee1bec66ba1235cfa7c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:54.219752 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ade89b68-f362-4ec3-a51a-40a60b6dc21d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f87a72599e4c4ee1bec66ba1235cfa7c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:54.228828 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ade89b68-f362-4ec3-a51a-40a60b6dc21d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f87a72599e4c4ee1bec66ba1235cfa7c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:54.229907 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ade89b68-f362-4ec3-a51a-40a60b6dc21d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f87a72599e4c4ee1bec66ba1235cfa7c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:54.247188 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ade89b68-f362-4ec3-a51a-40a60b6dc21d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f87a72599e4c4ee1bec66ba1235cfa7c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:54.255415 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1196/2391] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 19:58:54 2022] DELETE /identity/v3/users/f87a72599e4c4ee1bec66ba1235cfa7c => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:58:54.597787 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3eb3f5-5847-408d-8b5f-bb86806df605 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:54.599548 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ac3eb3f5-5847-408d-8b5f-bb86806df605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:58:54.621263 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3eb3f5-5847-408d-8b5f-bb86806df605 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:54.655285 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:58:54.699459 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6b2fbb-2c9e-4a37-a517-f5bcffd53351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:54.700413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b2fbb-2c9e-4a37-a517-f5bcffd53351 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:54.700693 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b2fbb-2c9e-4a37-a517-f5bcffd53351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:54.701024 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6b2fbb-2c9e-4a37-a517-f5bcffd53351 admin admin] PATH_INFO: `/v3/projects/0d780817942d4dafb886f6534534262a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:54.702287 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b2fbb-2c9e-4a37-a517-f5bcffd53351 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d780817942d4dafb886f6534534262a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:54.704315 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6b2fbb-2c9e-4a37-a517-f5bcffd53351 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:54.771241 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8a6b2fbb-2c9e-4a37-a517-f5bcffd53351 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d780817942d4dafb886f6534534262a', 'request_id': 'req-8a6b2fbb-2c9e-4a37-a517-f5bcffd53351'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:54.782033 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8a6b2fbb-2c9e-4a37-a517-f5bcffd53351 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d780817942d4dafb886f6534534262a', 'request_id': 'req-8a6b2fbb-2c9e-4a37-a517-f5bcffd53351'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:54.788841 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8a6b2fbb-2c9e-4a37-a517-f5bcffd53351 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d780817942d4dafb886f6534534262a', 'request_id': 'req-8a6b2fbb-2c9e-4a37-a517-f5bcffd53351'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:54.809308 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8a6b2fbb-2c9e-4a37-a517-f5bcffd53351 admin admin] The token cache is being invalidate because project 0d780817942d4dafb886f6534534262a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 19:58:54.809877 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8a6b2fbb-2c9e-4a37-a517-f5bcffd53351 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:54.810790 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8a6b2fbb-2c9e-4a37-a517-f5bcffd53351 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 19:58:54.812582 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1196/2392] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 19:58:54 2022] DELETE /identity/v3/projects/0d780817942d4dafb886f6534534262a => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:58:54.818904 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe24d3-e0d3-48f1-91be-cd0d23b5d6ed None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:54.819434 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe24d3-e0d3-48f1-91be-cd0d23b5d6ed None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:54.820004 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe24d3-e0d3-48f1-91be-cd0d23b5d6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:54.846612 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7fbe24d3-e0d3-48f1-91be-cd0d23b5d6ed None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:58:54.848335 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7fbe24d3-e0d3-48f1-91be-cd0d23b5d6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:58:54.876551 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:58:54.923734 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1197/2393] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:58:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 19:58:54.933526 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad308b21-c9cf-4245-b6f8-4826e82398a3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:54.934140 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad308b21-c9cf-4245-b6f8-4826e82398a3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:54.934298 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad308b21-c9cf-4245-b6f8-4826e82398a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:54.962707 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ad308b21-c9cf-4245-b6f8-4826e82398a3 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:58:54.964050 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ad308b21-c9cf-4245-b6f8-4826e82398a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:58:54.972655 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1197/2394] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:58:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:58:54.977764 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe24d3-e0d3-48f1-91be-cd0d23b5d6ed None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:54.988249 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe24d3-e0d3-48f1-91be-cd0d23b5d6ed None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:54.993423 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41db801-f981-4a75-85cf-5233d6e775bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:54.995265 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41db801-f981-4a75-85cf-5233d6e775bd admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:54.995265 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41db801-f981-4a75-85cf-5233d6e775bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:54.995265 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41db801-f981-4a75-85cf-5233d6e775bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:54.995721 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41db801-f981-4a75-85cf-5233d6e775bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 19:58:54.996035 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41db801-f981-4a75-85cf-5233d6e775bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:54.997707 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41db801-f981-4a75-85cf-5233d6e775bd admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:55.011177 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1198/2395] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 19:58:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 19:58:55.016143 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad308b21-c9cf-4245-b6f8-4826e82398a3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:55.027292 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad308b21-c9cf-4245-b6f8-4826e82398a3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:55.032251 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f3b76-0833-4c85-86a1-688ecaa50cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:55.032927 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f3b76-0833-4c85-86a1-688ecaa50cfb admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:55.033211 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f3b76-0833-4c85-86a1-688ecaa50cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:55.033507 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f3b76-0833-4c85-86a1-688ecaa50cfb admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:55.034376 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4f3b76-0833-4c85-86a1-688ecaa50cfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:55.036659 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4f3b76-0833-4c85-86a1-688ecaa50cfb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:55.070346 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1198/2396] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:58:55 2022] POST /identity/v3/projects => generated 393 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:58:55.074610 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41db801-f981-4a75-85cf-5233d6e775bd admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:55.085646 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41db801-f981-4a75-85cf-5233d6e775bd admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:55.168543 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c290cb-6ccb-416d-907f-d0b5dc38a18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:55.169344 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c290cb-6ccb-416d-907f-d0b5dc38a18f admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:55.169664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c290cb-6ccb-416d-907f-d0b5dc38a18f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:55.169947 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c290cb-6ccb-416d-907f-d0b5dc38a18f admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:55.170691 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c290cb-6ccb-416d-907f-d0b5dc38a18f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:55.172764 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c290cb-6ccb-416d-907f-d0b5dc38a18f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:55.223151 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1199/2397] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:58:55 2022] POST /identity/v3/users => generated 297 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:58:55.224683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f3b76-0833-4c85-86a1-688ecaa50cfb admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:55.237289 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f3b76-0833-4c85-86a1-688ecaa50cfb admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:55.245043 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895515f-d5c5-4c07-a840-510c075bd819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:55.245931 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895515f-d5c5-4c07-a840-510c075bd819 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:55.246230 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895515f-d5c5-4c07-a840-510c075bd819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:55.246515 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895515f-d5c5-4c07-a840-510c075bd819 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:55.247340 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7895515f-d5c5-4c07-a840-510c075bd819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:55.249553 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7895515f-d5c5-4c07-a840-510c075bd819 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:55.269144 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1199/2398] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:58:55.277624 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c290cb-6ccb-416d-907f-d0b5dc38a18f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:55.293318 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c290cb-6ccb-416d-907f-d0b5dc38a18f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:55.308669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938dada6-6339-4b27-9992-e241a21ea58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:55.309379 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938dada6-6339-4b27-9992-e241a21ea58e admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:55.309739 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938dada6-6339-4b27-9992-e241a21ea58e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:55.310025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938dada6-6339-4b27-9992-e241a21ea58e admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:55.310854 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938dada6-6339-4b27-9992-e241a21ea58e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:55.312668 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938dada6-6339-4b27-9992-e241a21ea58e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:55.333934 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895515f-d5c5-4c07-a840-510c075bd819 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:55.352152 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1200/2399] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:58:55.380376 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895515f-d5c5-4c07-a840-510c075bd819 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:55.388129 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8476687-0100-42cd-a905-a73ad5919e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:55.389155 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8476687-0100-42cd-a905-a73ad5919e8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:55.389442 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8476687-0100-42cd-a905-a73ad5919e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:55.390026 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8476687-0100-42cd-a905-a73ad5919e8f admin admin] PATH_INFO: `/v3/projects/db0483be7f11466cabbb3c39a01749a5/users/be58ac7a8de3435194440c9587e473be/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:55.418691 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8476687-0100-42cd-a905-a73ad5919e8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=db0483be7f11466cabbb3c39a01749a5, user_id=be58ac7a8de3435194440c9587e473be, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:55.422917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8476687-0100-42cd-a905-a73ad5919e8f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:55.449471 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1200/2400] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:58:55 2022] PUT /identity/v3/projects/db0483be7f11466cabbb3c39a01749a5/users/be58ac7a8de3435194440c9587e473be/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:58:55.462241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959c29d-ecd2-41cc-adf5-19bbdb8a65ca None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:55.462520 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959c29d-ecd2-41cc-adf5-19bbdb8a65ca None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:55.462791 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a959c29d-ecd2-41cc-adf5-19bbdb8a65ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:55.504108 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a959c29d-ecd2-41cc-adf5-19bbdb8a65ca None None] MFA Rules not processed for user `be58ac7a8de3435194440c9587e473be`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:58:55.506180 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a959c29d-ecd2-41cc-adf5-19bbdb8a65ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:58:55.609883 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1201/2401] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:58:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:58:55.615460 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8476687-0100-42cd-a905-a73ad5919e8f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:55.684615 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:58:55.699745 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8476687-0100-42cd-a905-a73ad5919e8f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:55.706473 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416376b6-a274-41cf-a82c-9dd1adbdb26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:55.707402 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416376b6-a274-41cf-a82c-9dd1adbdb26c admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:55.708518 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416376b6-a274-41cf-a82c-9dd1adbdb26c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:55.709023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416376b6-a274-41cf-a82c-9dd1adbdb26c admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:55.709813 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416376b6-a274-41cf-a82c-9dd1adbdb26c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:55.712141 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416376b6-a274-41cf-a82c-9dd1adbdb26c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:55.745295 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1201/2402] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 19:58:55 2022] POST /identity/v3/projects => generated 393 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:58:55.759601 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a959c29d-ecd2-41cc-adf5-19bbdb8a65ca None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:55.772576 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a959c29d-ecd2-41cc-adf5-19bbdb8a65ca None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:55.816851 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:58:55.904683 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d384e415-c02d-478f-bf4b-80b58b95670a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:55.905884 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384e415-c02d-478f-bf4b-80b58b95670a admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:55.906189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384e415-c02d-478f-bf4b-80b58b95670a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:55.906484 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384e415-c02d-478f-bf4b-80b58b95670a admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:55.907389 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d384e415-c02d-478f-bf4b-80b58b95670a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:55.910623 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d384e415-c02d-478f-bf4b-80b58b95670a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:55.955234 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1202/2403] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 19:58:55 2022] POST /identity/v3/users => generated 303 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 19:58:55.962133 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416376b6-a274-41cf-a82c-9dd1adbdb26c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:55.978943 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416376b6-a274-41cf-a82c-9dd1adbdb26c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:55.989765 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205407b-aeff-445c-a8fb-f59cbb9a72f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:55.990780 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205407b-aeff-445c-a8fb-f59cbb9a72f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:55.991069 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205407b-aeff-445c-a8fb-f59cbb9a72f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:55.991353 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205407b-aeff-445c-a8fb-f59cbb9a72f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:55.992159 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4205407b-aeff-445c-a8fb-f59cbb9a72f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:55.993971 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4205407b-aeff-445c-a8fb-f59cbb9a72f7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:56.020452 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1202/2404] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:58:56.026014 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d384e415-c02d-478f-bf4b-80b58b95670a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:56.044699 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d384e415-c02d-478f-bf4b-80b58b95670a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:56.055673 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233eb0e1-2234-4bd2-ac16-f4b9cddf7e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:56.058227 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233eb0e1-2234-4bd2-ac16-f4b9cddf7e05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:56.058630 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233eb0e1-2234-4bd2-ac16-f4b9cddf7e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:56.059026 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233eb0e1-2234-4bd2-ac16-f4b9cddf7e05 admin admin] PATH_INFO: `/v3/projects/144c02a9b3ad44a2b7b2e94656024f27/users/d2a5c017f8044b5cb4a120a46659c0fb/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:56.092898 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233eb0e1-2234-4bd2-ac16-f4b9cddf7e05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=144c02a9b3ad44a2b7b2e94656024f27, user_id=d2a5c017f8044b5cb4a120a46659c0fb, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:56.096766 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233eb0e1-2234-4bd2-ac16-f4b9cddf7e05 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:56.122043 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1203/2405] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:58:56 2022] PUT /identity/v3/projects/144c02a9b3ad44a2b7b2e94656024f27/users/d2a5c017f8044b5cb4a120a46659c0fb/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 19:58:56.130049 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205407b-aeff-445c-a8fb-f59cbb9a72f7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:56.143922 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205407b-aeff-445c-a8fb-f59cbb9a72f7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:56.239676 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676f16c-ab8c-424c-8354-1a714d6d9a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:56.241258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676f16c-ab8c-424c-8354-1a714d6d9a3a admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:56.241621 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676f16c-ab8c-424c-8354-1a714d6d9a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:56.241907 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676f16c-ab8c-424c-8354-1a714d6d9a3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:56.242868 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f676f16c-ab8c-424c-8354-1a714d6d9a3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:56.244623 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f676f16c-ab8c-424c-8354-1a714d6d9a3a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:56.257875 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1203/2406] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 19:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 19:58:56.263939 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233eb0e1-2234-4bd2-ac16-f4b9cddf7e05 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:58:56.282761 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233eb0e1-2234-4bd2-ac16-f4b9cddf7e05 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:58:56.288775 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271d787-d384-4521-992e-f02ce83daaeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:58:56.289722 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271d787-d384-4521-992e-f02ce83daaeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:56.289947 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271d787-d384-4521-992e-f02ce83daaeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:56.290230 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271d787-d384-4521-992e-f02ce83daaeb admin admin] PATH_INFO: `/v3/projects/144c02a9b3ad44a2b7b2e94656024f27/users/d2a5c017f8044b5cb4a120a46659c0fb/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:56.292295 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7271d787-d384-4521-992e-f02ce83daaeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=144c02a9b3ad44a2b7b2e94656024f27, user_id=d2a5c017f8044b5cb4a120a46659c0fb, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:58:56.294965 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7271d787-d384-4521-992e-f02ce83daaeb admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:58:56.309064 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1204/2407] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 19:58:56 2022] PUT /identity/v3/projects/144c02a9b3ad44a2b7b2e94656024f27/users/d2a5c017f8044b5cb4a120a46659c0fb/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 19:58:56.320792 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ca6c3-7f1a-455f-ac9c-df18a3e74be7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:58:56.321086 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ca6c3-7f1a-455f-ac9c-df18a3e74be7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:58:56.321371 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ca6c3-7f1a-455f-ac9c-df18a3e74be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:58:56.356118 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-130ca6c3-7f1a-455f-ac9c-df18a3e74be7 None None] MFA Rules not processed for user `d2a5c017f8044b5cb4a120a46659c0fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 19:58:56.357356 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-130ca6c3-7f1a-455f-ac9c-df18a3e74be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:58:56.466945 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1204/2408] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 19:58:56 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 19:59:06.304049 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=32,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:59:06.463591 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=77,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 19:59:13.062770 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271d787-d384-4521-992e-f02ce83daaeb admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 19:59:13.064573 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7271d787-d384-4521-992e-f02ce83daaeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:59:13.154244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271d787-d384-4521-992e-f02ce83daaeb admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 19:59:13.160669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ca90b-6caa-4c5f-812f-0b59e75f2c7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 19:59:13.161339 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ca90b-6caa-4c5f-812f-0b59e75f2c7a service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 19:59:13.161631 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ca90b-6caa-4c5f-812f-0b59e75f2c7a service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 19:59:13.161913 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ca90b-6caa-4c5f-812f-0b59e75f2c7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 19:59:13.163925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ab8ca90b-6caa-4c5f-812f-0b59e75f2c7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 19:59:13.251788 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8ca90b-6caa-4c5f-812f-0b59e75f2c7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 19:59:13.253502 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8ca90b-6caa-4c5f-812f-0b59e75f2c7a service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 19:59:13.274015 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1205/2409] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 19:59:13 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 19:59:23.270405 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=69 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:00:15.778873 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ca6c3-7f1a-455f-ac9c-df18a3e74be7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:00:15.780929 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-130ca6c3-7f1a-455f-ac9c-df18a3e74be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:00:15.841549 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ca6c3-7f1a-455f-ac9c-df18a3e74be7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:00:15.845385 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f497ab0-856c-43b1-babe-d9249f168d17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:00:15.846931 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f497ab0-856c-43b1-babe-d9249f168d17 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:00:15.847418 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f497ab0-856c-43b1-babe-d9249f168d17 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:00:15.847962 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f497ab0-856c-43b1-babe-d9249f168d17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:00:15.850470 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9f497ab0-856c-43b1-babe-d9249f168d17 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:00:15.945120 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f497ab0-856c-43b1-babe-d9249f168d17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:00:15.946779 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f497ab0-856c-43b1-babe-d9249f168d17 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:00:15.969381 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1205/2410] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:00:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:00:16.017198 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ca90b-6caa-4c5f-812f-0b59e75f2c7a service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:00:16.031465 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ca90b-6caa-4c5f-812f-0b59e75f2c7a service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:00:16.036417 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037ca344-e0ce-4086-932c-fa3d5095cdbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:00:16.037072 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ca344-e0ce-4086-932c-fa3d5095cdbe service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:00:16.037361 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ca344-e0ce-4086-932c-fa3d5095cdbe service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:00:16.037643 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ca344-e0ce-4086-932c-fa3d5095cdbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:00:16.049829 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037ca344-e0ce-4086-932c-fa3d5095cdbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:00:16.051295 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037ca344-e0ce-4086-932c-fa3d5095cdbe service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:00:16.062015 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1206/2411] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:00:16 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:00:25.966666 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=61 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:00:26.057164 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:00:53.128537 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f497ab0-856c-43b1-babe-d9249f168d17 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:00:53.130717 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9f497ab0-856c-43b1-babe-d9249f168d17 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:00:53.148523 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f497ab0-856c-43b1-babe-d9249f168d17 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:00:53.153552 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a3b2f-7302-4a3f-8b92-da46a8baefcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:00:53.154215 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a3b2f-7302-4a3f-8b92-da46a8baefcf service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:00:53.154530 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a3b2f-7302-4a3f-8b92-da46a8baefcf service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:00:53.154809 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a3b2f-7302-4a3f-8b92-da46a8baefcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:00:53.220579 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7a3b2f-7302-4a3f-8b92-da46a8baefcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:00:53.222070 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7a3b2f-7302-4a3f-8b92-da46a8baefcf service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:00:53.231296 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1206/2412] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:00:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:01:03.227128 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=28 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:01:15.193859 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037ca344-e0ce-4086-932c-fa3d5095cdbe service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:15.207080 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037ca344-e0ce-4086-932c-fa3d5095cdbe service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:15.211452 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76772c28-c327-4a83-83f8-e05895bccd0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:15.212263 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76772c28-c327-4a83-83f8-e05895bccd0c service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:15.212617 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76772c28-c327-4a83-83f8-e05895bccd0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:15.212968 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76772c28-c327-4a83-83f8-e05895bccd0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:15.214903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-76772c28-c327-4a83-83f8-e05895bccd0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:15.230289 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76772c28-c327-4a83-83f8-e05895bccd0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:15.231764 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76772c28-c327-4a83-83f8-e05895bccd0c service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:15.244407 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1207/2413] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:01:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:01:15.251861 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a3b2f-7302-4a3f-8b92-da46a8baefcf service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:15.261804 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a3b2f-7302-4a3f-8b92-da46a8baefcf service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:15.266312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28227329-60e9-4cf1-95df-7936e441766c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:15.267004 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28227329-60e9-4cf1-95df-7936e441766c service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:15.267293 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28227329-60e9-4cf1-95df-7936e441766c service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:15.267618 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28227329-60e9-4cf1-95df-7936e441766c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:15.269495 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-28227329-60e9-4cf1-95df-7936e441766c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:15.335549 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28227329-60e9-4cf1-95df-7936e441766c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:15.337014 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28227329-60e9-4cf1-95df-7936e441766c service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:15.347238 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1207/2414] 10.222.0.18 () {60 vars in 1396 bytes} [Mon Jul 25 20:01:15 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:01:16.514135 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76772c28-c327-4a83-83f8-e05895bccd0c service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:16.516428 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-76772c28-c327-4a83-83f8-e05895bccd0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:16.538486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76772c28-c327-4a83-83f8-e05895bccd0c service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:16.547976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbab104-065a-4338-a2d0-eb8cb727d4f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:16.548919 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbab104-065a-4338-a2d0-eb8cb727d4f6 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:16.549398 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbab104-065a-4338-a2d0-eb8cb727d4f6 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:16.549974 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbab104-065a-4338-a2d0-eb8cb727d4f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:16.552069 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fcbab104-065a-4338-a2d0-eb8cb727d4f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:16.583666 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbab104-065a-4338-a2d0-eb8cb727d4f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:16.585803 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbab104-065a-4338-a2d0-eb8cb727d4f6 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:16.610424 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1208/2415] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:01:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:01:16.807533 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28227329-60e9-4cf1-95df-7936e441766c service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:16.921318 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28227329-60e9-4cf1-95df-7936e441766c service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:16.926978 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a9d712-0b6d-4317-9fc6-7b6b4be09915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:16.927987 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9d712-0b6d-4317-9fc6-7b6b4be09915 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:16.928292 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9d712-0b6d-4317-9fc6-7b6b4be09915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:16.928768 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9d712-0b6d-4317-9fc6-7b6b4be09915 admin admin] PATH_INFO: `/v3/users/be58ac7a8de3435194440c9587e473be` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:16.930214 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a9d712-0b6d-4317-9fc6-7b6b4be09915 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be58ac7a8de3435194440c9587e473be)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:16.932557 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a9d712-0b6d-4317-9fc6-7b6b4be09915 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:16.993743 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-55a9d712-0b6d-4317-9fc6-7b6b4be09915 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be58ac7a8de3435194440c9587e473be'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:17.009226 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-55a9d712-0b6d-4317-9fc6-7b6b4be09915 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be58ac7a8de3435194440c9587e473be'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:17.023366 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-55a9d712-0b6d-4317-9fc6-7b6b4be09915 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be58ac7a8de3435194440c9587e473be'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:17.025571 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-55a9d712-0b6d-4317-9fc6-7b6b4be09915 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be58ac7a8de3435194440c9587e473be'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:17.027714 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-55a9d712-0b6d-4317-9fc6-7b6b4be09915 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be58ac7a8de3435194440c9587e473be'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:17.065237 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1208/2416] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:01:16 2022] DELETE /identity/v3/users/be58ac7a8de3435194440c9587e473be => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:01:17.072584 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760afec-af89-4f3c-b4fc-a97c7b7ad4c0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:17.073421 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760afec-af89-4f3c-b4fc-a97c7b7ad4c0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:17.074350 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760afec-af89-4f3c-b4fc-a97c7b7ad4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:17.105684 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4760afec-af89-4f3c-b4fc-a97c7b7ad4c0 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:01:17.107146 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4760afec-af89-4f3c-b4fc-a97c7b7ad4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:17.231904 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1209/2417] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:01:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:01:17.244755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a9d712-0b6d-4317-9fc6-7b6b4be09915 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:17.246305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-55a9d712-0b6d-4317-9fc6-7b6b4be09915 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:17.274635 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:01:17.327062 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a9d712-0b6d-4317-9fc6-7b6b4be09915 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:17.331717 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ffe3ed-be93-4910-adf2-628b676bdb34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:17.332474 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ffe3ed-be93-4910-adf2-628b676bdb34 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:17.332834 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ffe3ed-be93-4910-adf2-628b676bdb34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:17.333277 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ffe3ed-be93-4910-adf2-628b676bdb34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:17.350457 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ffe3ed-be93-4910-adf2-628b676bdb34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:17.352307 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ffe3ed-be93-4910-adf2-628b676bdb34 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:17.363927 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1209/2418] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:01:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:01:17.859040 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760afec-af89-4f3c-b4fc-a97c7b7ad4c0 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:17.861292 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4760afec-af89-4f3c-b4fc-a97c7b7ad4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:17.875811 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760afec-af89-4f3c-b4fc-a97c7b7ad4c0 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:17.883353 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac0c368-26e9-4065-8178-f78d0890af79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:17.884342 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0c368-26e9-4065-8178-f78d0890af79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:17.884652 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0c368-26e9-4065-8178-f78d0890af79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:17.884932 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0c368-26e9-4065-8178-f78d0890af79 admin admin] PATH_INFO: `/v3/projects/db0483be7f11466cabbb3c39a01749a5` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:17.886866 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac0c368-26e9-4065-8178-f78d0890af79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db0483be7f11466cabbb3c39a01749a5)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:17.888931 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac0c368-26e9-4065-8178-f78d0890af79 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:17.960762 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2ac0c368-26e9-4065-8178-f78d0890af79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db0483be7f11466cabbb3c39a01749a5', 'request_id': 'req-2ac0c368-26e9-4065-8178-f78d0890af79'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:17.970448 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2ac0c368-26e9-4065-8178-f78d0890af79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db0483be7f11466cabbb3c39a01749a5', 'request_id': 'req-2ac0c368-26e9-4065-8178-f78d0890af79'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:17.977130 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2ac0c368-26e9-4065-8178-f78d0890af79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db0483be7f11466cabbb3c39a01749a5', 'request_id': 'req-2ac0c368-26e9-4065-8178-f78d0890af79'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:17.989620 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2ac0c368-26e9-4065-8178-f78d0890af79 admin admin] The token cache is being invalidate because project db0483be7f11466cabbb3c39a01749a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:01:17.990004 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2ac0c368-26e9-4065-8178-f78d0890af79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:17.990926 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2ac0c368-26e9-4065-8178-f78d0890af79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:17.992490 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1210/2419] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:01:17 2022] DELETE /identity/v3/projects/db0483be7f11466cabbb3c39a01749a5 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:01:17.996501 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ffe3ed-be93-4910-adf2-628b676bdb34 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:18.006865 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ffe3ed-be93-4910-adf2-628b676bdb34 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:18.071675 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e03367-c356-4e1b-a388-c5af675abdf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:18.072550 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e03367-c356-4e1b-a388-c5af675abdf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:18.072836 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e03367-c356-4e1b-a388-c5af675abdf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:18.073114 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e03367-c356-4e1b-a388-c5af675abdf6 admin admin] PATH_INFO: `/v3/users/d2a5c017f8044b5cb4a120a46659c0fb` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:18.074325 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e03367-c356-4e1b-a388-c5af675abdf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2a5c017f8044b5cb4a120a46659c0fb)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:18.076361 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e03367-c356-4e1b-a388-c5af675abdf6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:18.135227 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-72e03367-c356-4e1b-a388-c5af675abdf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2a5c017f8044b5cb4a120a46659c0fb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:18.143150 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-72e03367-c356-4e1b-a388-c5af675abdf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2a5c017f8044b5cb4a120a46659c0fb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:18.159427 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-72e03367-c356-4e1b-a388-c5af675abdf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2a5c017f8044b5cb4a120a46659c0fb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:18.160766 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-72e03367-c356-4e1b-a388-c5af675abdf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2a5c017f8044b5cb4a120a46659c0fb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:18.161516 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-72e03367-c356-4e1b-a388-c5af675abdf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2a5c017f8044b5cb4a120a46659c0fb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:18.192903 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1210/2420] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:01:17 2022] DELETE /identity/v3/users/d2a5c017f8044b5cb4a120a46659c0fb => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:01:18.522515 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac0c368-26e9-4065-8178-f78d0890af79 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:18.524937 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2ac0c368-26e9-4065-8178-f78d0890af79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:18.541021 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac0c368-26e9-4065-8178-f78d0890af79 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:18.602991 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:01:18.611407 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddaa441-ed45-48f6-8a48-49fcc39eb51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:18.612363 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddaa441-ed45-48f6-8a48-49fcc39eb51d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:18.612906 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddaa441-ed45-48f6-8a48-49fcc39eb51d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:18.613372 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddaa441-ed45-48f6-8a48-49fcc39eb51d admin admin] PATH_INFO: `/v3/projects/144c02a9b3ad44a2b7b2e94656024f27` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:18.615363 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddaa441-ed45-48f6-8a48-49fcc39eb51d admin admin] RBAC: Authorizing `identity:delete_project(project_id=144c02a9b3ad44a2b7b2e94656024f27)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:18.617986 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddaa441-ed45-48f6-8a48-49fcc39eb51d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:18.666124 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2ddaa441-ed45-48f6-8a48-49fcc39eb51d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '144c02a9b3ad44a2b7b2e94656024f27', 'request_id': 'req-2ddaa441-ed45-48f6-8a48-49fcc39eb51d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:18.678945 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2ddaa441-ed45-48f6-8a48-49fcc39eb51d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '144c02a9b3ad44a2b7b2e94656024f27', 'request_id': 'req-2ddaa441-ed45-48f6-8a48-49fcc39eb51d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:18.685584 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2ddaa441-ed45-48f6-8a48-49fcc39eb51d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '144c02a9b3ad44a2b7b2e94656024f27', 'request_id': 'req-2ddaa441-ed45-48f6-8a48-49fcc39eb51d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:18.698329 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2ddaa441-ed45-48f6-8a48-49fcc39eb51d admin admin] The token cache is being invalidate because project 144c02a9b3ad44a2b7b2e94656024f27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:01:18.698624 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2ddaa441-ed45-48f6-8a48-49fcc39eb51d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:18.699295 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2ddaa441-ed45-48f6-8a48-49fcc39eb51d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:18.701060 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1211/2421] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:01:18 2022] DELETE /identity/v3/projects/144c02a9b3ad44a2b7b2e94656024f27 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:01:18.706411 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cffd2b-7f4f-4346-bcda-cb62207784fb None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:18.706773 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cffd2b-7f4f-4346-bcda-cb62207784fb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:18.707059 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cffd2b-7f4f-4346-bcda-cb62207784fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:18.730701 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-93cffd2b-7f4f-4346-bcda-cb62207784fb None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:01:18.731969 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-93cffd2b-7f4f-4346-bcda-cb62207784fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:18.769279 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:01:18.799971 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1211/2422] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:01:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:01:18.808097 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9010ca7-c5ef-44b8-a773-43d63bf72b0f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:18.808540 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9010ca7-c5ef-44b8-a773-43d63bf72b0f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:18.808894 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9010ca7-c5ef-44b8-a773-43d63bf72b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:18.833801 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e9010ca7-c5ef-44b8-a773-43d63bf72b0f None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:01:18.835483 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e9010ca7-c5ef-44b8-a773-43d63bf72b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:18.846710 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1212/2423] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:01:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:01:18.851373 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cffd2b-7f4f-4346-bcda-cb62207784fb None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:18.867406 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cffd2b-7f4f-4346-bcda-cb62207784fb None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:18.872981 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dece27b-d780-49cc-b06b-c70b7a0812b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:18.873660 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dece27b-d780-49cc-b06b-c70b7a0812b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:18.873967 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dece27b-d780-49cc-b06b-c70b7a0812b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:18.874235 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dece27b-d780-49cc-b06b-c70b7a0812b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:18.875098 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dece27b-d780-49cc-b06b-c70b7a0812b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 20:01:18.875500 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dece27b-d780-49cc-b06b-c70b7a0812b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:18.877203 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dece27b-d780-49cc-b06b-c70b7a0812b3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:18.891148 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1212/2424] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 20:01:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 20:01:18.895881 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9010ca7-c5ef-44b8-a773-43d63bf72b0f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:18.909089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9010ca7-c5ef-44b8-a773-43d63bf72b0f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:18.915206 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c28cdd-fa9e-4399-94b9-7dec62cbe912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:18.915937 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c28cdd-fa9e-4399-94b9-7dec62cbe912 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:18.916279 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c28cdd-fa9e-4399-94b9-7dec62cbe912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:18.916558 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c28cdd-fa9e-4399-94b9-7dec62cbe912 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:18.917676 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c28cdd-fa9e-4399-94b9-7dec62cbe912 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:18.920030 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c28cdd-fa9e-4399-94b9-7dec62cbe912 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:18.955058 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1213/2425] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:01:18 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:01:18.962996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dece27b-d780-49cc-b06b-c70b7a0812b3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:18.976321 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dece27b-d780-49cc-b06b-c70b7a0812b3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:19.059076 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ee8d4-b66b-468d-863d-ad58273c9fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:19.059076 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ee8d4-b66b-468d-863d-ad58273c9fc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:19.059076 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ee8d4-b66b-468d-863d-ad58273c9fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:19.059076 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ee8d4-b66b-468d-863d-ad58273c9fc6 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:19.059904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ee8d4-b66b-468d-863d-ad58273c9fc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:19.062068 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ee8d4-b66b-468d-863d-ad58273c9fc6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:19.088233 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1213/2426] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:01:18 2022] POST /identity/v3/users => generated 292 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:01:19.092313 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c28cdd-fa9e-4399-94b9-7dec62cbe912 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:19.104527 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c28cdd-fa9e-4399-94b9-7dec62cbe912 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:19.115909 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d700f5c-3b34-4cea-9134-ec256318ccd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:19.117456 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d700f5c-3b34-4cea-9134-ec256318ccd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:19.118347 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d700f5c-3b34-4cea-9134-ec256318ccd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:19.118770 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d700f5c-3b34-4cea-9134-ec256318ccd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:19.119717 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d700f5c-3b34-4cea-9134-ec256318ccd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:19.121584 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d700f5c-3b34-4cea-9134-ec256318ccd6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:19.133871 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1214/2427] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:01:19 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:01:19.137970 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ee8d4-b66b-468d-863d-ad58273c9fc6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:19.148914 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ee8d4-b66b-468d-863d-ad58273c9fc6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:19.154115 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8df155-e60b-4956-aeed-6ad80b1317ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:19.154789 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8df155-e60b-4956-aeed-6ad80b1317ea admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:19.155079 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8df155-e60b-4956-aeed-6ad80b1317ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:19.155361 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8df155-e60b-4956-aeed-6ad80b1317ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:19.156191 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8df155-e60b-4956-aeed-6ad80b1317ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:19.158299 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8df155-e60b-4956-aeed-6ad80b1317ea admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:19.170213 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1214/2428] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:01:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 20:01:19.174731 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d700f5c-3b34-4cea-9134-ec256318ccd6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:19.185121 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d700f5c-3b34-4cea-9134-ec256318ccd6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:19.191363 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2a79d3-4fc6-48e4-adb0-86f16d0922cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:19.192322 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a79d3-4fc6-48e4-adb0-86f16d0922cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:19.192605 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a79d3-4fc6-48e4-adb0-86f16d0922cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:19.192886 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a79d3-4fc6-48e4-adb0-86f16d0922cf admin admin] PATH_INFO: `/v3/projects/b27b2c7d706c4999bdb520e68efb178b/users/70add6bc7bfd4667932073040c3ba4ba/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:19.213875 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2a79d3-4fc6-48e4-adb0-86f16d0922cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b27b2c7d706c4999bdb520e68efb178b, user_id=70add6bc7bfd4667932073040c3ba4ba, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:19.216740 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2a79d3-4fc6-48e4-adb0-86f16d0922cf admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:19.238467 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1215/2429] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:01:19 2022] PUT /identity/v3/projects/b27b2c7d706c4999bdb520e68efb178b/users/70add6bc7bfd4667932073040c3ba4ba/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:01:19.308793 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ee416-2838-48bc-9866-b06f6d243eb6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:19.309156 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ee416-2838-48bc-9866-b06f6d243eb6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:19.309394 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ee416-2838-48bc-9866-b06f6d243eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:19.339488 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-943ee416-2838-48bc-9866-b06f6d243eb6 None None] MFA Rules not processed for user `70add6bc7bfd4667932073040c3ba4ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:01:19.340801 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-943ee416-2838-48bc-9866-b06f6d243eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:19.415151 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1215/2430] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:01:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:01:19.416487 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2a79d3-4fc6-48e4-adb0-86f16d0922cf admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:19.488125 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2a79d3-4fc6-48e4-adb0-86f16d0922cf admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:19.493373 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa8f54-b468-4c47-8413-0ca84ddc9062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:19.494506 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa8f54-b468-4c47-8413-0ca84ddc9062 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:19.494674 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa8f54-b468-4c47-8413-0ca84ddc9062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:19.494873 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa8f54-b468-4c47-8413-0ca84ddc9062 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:19.495599 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa8f54-b468-4c47-8413-0ca84ddc9062 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:19.497577 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa8f54-b468-4c47-8413-0ca84ddc9062 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:19.538905 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1216/2431] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:01:19 2022] POST /identity/v3/projects => generated 381 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:01:19.543038 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ee416-2838-48bc-9866-b06f6d243eb6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:19.555758 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ee416-2838-48bc-9866-b06f6d243eb6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:19.613448 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:01:19.649113 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5968e4e-f692-49f1-b3bc-7546746983ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:19.649980 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5968e4e-f692-49f1-b3bc-7546746983ce admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:19.650328 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5968e4e-f692-49f1-b3bc-7546746983ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:19.650651 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5968e4e-f692-49f1-b3bc-7546746983ce admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:19.651402 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5968e4e-f692-49f1-b3bc-7546746983ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:19.654456 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5968e4e-f692-49f1-b3bc-7546746983ce admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:19.686862 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1216/2432] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:01:19 2022] POST /identity/v3/users => generated 297 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 20:01:19.692031 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa8f54-b468-4c47-8413-0ca84ddc9062 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:19.706722 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa8f54-b468-4c47-8413-0ca84ddc9062 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:19.714916 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38d7f5-2669-4d5e-bdcd-c9bcc5457945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:19.714916 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38d7f5-2669-4d5e-bdcd-c9bcc5457945 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:19.714916 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38d7f5-2669-4d5e-bdcd-c9bcc5457945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:19.714916 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38d7f5-2669-4d5e-bdcd-c9bcc5457945 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:19.715325 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38d7f5-2669-4d5e-bdcd-c9bcc5457945 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:19.716426 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38d7f5-2669-4d5e-bdcd-c9bcc5457945 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:19.718966 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:01:19.730325 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1217/2433] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:01:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:01:19.734278 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5968e4e-f692-49f1-b3bc-7546746983ce admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:19.747372 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5968e4e-f692-49f1-b3bc-7546746983ce admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:19.758249 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa0443-4469-4d05-8db1-3d746c7c9fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:19.759159 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa0443-4469-4d05-8db1-3d746c7c9fb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:19.759442 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa0443-4469-4d05-8db1-3d746c7c9fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:19.759766 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa0443-4469-4d05-8db1-3d746c7c9fb3 admin admin] PATH_INFO: `/v3/projects/ccb031845f8243e895f833badeba8464/users/74cb656c25824d99ab50619b9666dde6/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:19.780058 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa0443-4469-4d05-8db1-3d746c7c9fb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccb031845f8243e895f833badeba8464, user_id=74cb656c25824d99ab50619b9666dde6, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:19.782827 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa0443-4469-4d05-8db1-3d746c7c9fb3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:19.798113 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1217/2434] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:01:19 2022] PUT /identity/v3/projects/ccb031845f8243e895f833badeba8464/users/74cb656c25824d99ab50619b9666dde6/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 20:01:19.802160 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38d7f5-2669-4d5e-bdcd-c9bcc5457945 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:19.815721 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38d7f5-2669-4d5e-bdcd-c9bcc5457945 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:19.886234 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd6e59-373b-47eb-9266-feffaaab31d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:19.886925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd6e59-373b-47eb-9266-feffaaab31d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:19.887256 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd6e59-373b-47eb-9266-feffaaab31d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:19.888187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd6e59-373b-47eb-9266-feffaaab31d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:19.888406 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fd6e59-373b-47eb-9266-feffaaab31d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:19.889995 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fd6e59-373b-47eb-9266-feffaaab31d1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:19.903802 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1218/2435] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:01:19 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:01:19.908194 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa0443-4469-4d05-8db1-3d746c7c9fb3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:19.921865 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa0443-4469-4d05-8db1-3d746c7c9fb3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:19.927851 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601cae5-f529-44c7-b6ce-54217225e2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:19.928780 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601cae5-f529-44c7-b6ce-54217225e2d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:19.929184 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601cae5-f529-44c7-b6ce-54217225e2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:19.929468 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601cae5-f529-44c7-b6ce-54217225e2d7 admin admin] PATH_INFO: `/v3/projects/ccb031845f8243e895f833badeba8464/users/74cb656c25824d99ab50619b9666dde6/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:19.931605 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f601cae5-f529-44c7-b6ce-54217225e2d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccb031845f8243e895f833badeba8464, user_id=74cb656c25824d99ab50619b9666dde6, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:19.934370 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f601cae5-f529-44c7-b6ce-54217225e2d7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:19.947032 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1218/2436] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:01:19 2022] PUT /identity/v3/projects/ccb031845f8243e895f833badeba8464/users/74cb656c25824d99ab50619b9666dde6/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 20:01:19.961882 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d4d23-399d-490e-8daa-a9572d274e6f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:19.962193 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d4d23-399d-490e-8daa-a9572d274e6f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:19.962480 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d4d23-399d-490e-8daa-a9572d274e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:20.007952 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2c2d4d23-399d-490e-8daa-a9572d274e6f None None] MFA Rules not processed for user `74cb656c25824d99ab50619b9666dde6`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:01:20.009188 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2c2d4d23-399d-490e-8daa-a9572d274e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:20.147723 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1219/2437] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:01:19 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:01:20.165423 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601cae5-f529-44c7-b6ce-54217225e2d7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:20.168025 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f601cae5-f529-44c7-b6ce-54217225e2d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:20.250525 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601cae5-f529-44c7-b6ce-54217225e2d7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:20.255302 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef8dc7-b3ca-4115-86bb-815e0da83e2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:20.256395 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef8dc7-b3ca-4115-86bb-815e0da83e2e service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:20.256903 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef8dc7-b3ca-4115-86bb-815e0da83e2e service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:20.257393 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef8dc7-b3ca-4115-86bb-815e0da83e2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:20.276024 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ef8dc7-b3ca-4115-86bb-815e0da83e2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:20.277625 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ef8dc7-b3ca-4115-86bb-815e0da83e2e service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:20.291732 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1219/2438] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:01:20 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:01:20.413842 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d4d23-399d-490e-8daa-a9572d274e6f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:20.429413 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d4d23-399d-490e-8daa-a9572d274e6f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:20.433487 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c3bd9d-5d0d-4719-a14d-9ed1311d3642 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:20.434257 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3bd9d-5d0d-4719-a14d-9ed1311d3642 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:20.434547 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3bd9d-5d0d-4719-a14d-9ed1311d3642 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:20.434867 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3bd9d-5d0d-4719-a14d-9ed1311d3642 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:20.524722 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c3bd9d-5d0d-4719-a14d-9ed1311d3642 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:20.527117 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c3bd9d-5d0d-4719-a14d-9ed1311d3642 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:20.542783 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1220/2439] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:01:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:01:20.933208 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef8dc7-b3ca-4115-86bb-815e0da83e2e service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:21.004194 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef8dc7-b3ca-4115-86bb-815e0da83e2e service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:21.009725 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37885a95-fdf2-4c40-b50f-7932f45dcd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:21.010616 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37885a95-fdf2-4c40-b50f-7932f45dcd3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:21.010912 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37885a95-fdf2-4c40-b50f-7932f45dcd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:21.011195 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37885a95-fdf2-4c40-b50f-7932f45dcd3c admin admin] PATH_INFO: `/v3/users/70add6bc7bfd4667932073040c3ba4ba` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:21.012459 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37885a95-fdf2-4c40-b50f-7932f45dcd3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=70add6bc7bfd4667932073040c3ba4ba)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:21.015253 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37885a95-fdf2-4c40-b50f-7932f45dcd3c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:21.054465 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:01:21.070304 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-37885a95-fdf2-4c40-b50f-7932f45dcd3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70add6bc7bfd4667932073040c3ba4ba'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:21.078449 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-37885a95-fdf2-4c40-b50f-7932f45dcd3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70add6bc7bfd4667932073040c3ba4ba'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:21.086643 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-37885a95-fdf2-4c40-b50f-7932f45dcd3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70add6bc7bfd4667932073040c3ba4ba'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:21.087595 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-37885a95-fdf2-4c40-b50f-7932f45dcd3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70add6bc7bfd4667932073040c3ba4ba'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:21.088353 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-37885a95-fdf2-4c40-b50f-7932f45dcd3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70add6bc7bfd4667932073040c3ba4ba'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:21.112745 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1220/2440] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:01:20 2022] DELETE /identity/v3/users/70add6bc7bfd4667932073040c3ba4ba => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:01:21.119311 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fc3af-3ece-4354-8f6d-789f5e11ebd2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:21.119842 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fc3af-3ece-4354-8f6d-789f5e11ebd2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:21.120166 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42fc3af-3ece-4354-8f6d-789f5e11ebd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:21.143574 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f42fc3af-3ece-4354-8f6d-789f5e11ebd2 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:01:21.145488 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f42fc3af-3ece-4354-8f6d-789f5e11ebd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:21.163973 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:01:21.245112 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1221/2441] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:01:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:01:21.256192 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37885a95-fdf2-4c40-b50f-7932f45dcd3c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:21.258521 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-37885a95-fdf2-4c40-b50f-7932f45dcd3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:21.337445 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37885a95-fdf2-4c40-b50f-7932f45dcd3c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:21.345964 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27691810-53a8-4ce2-ad6b-976404854475 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:21.347013 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27691810-53a8-4ce2-ad6b-976404854475 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:21.347540 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27691810-53a8-4ce2-ad6b-976404854475 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:21.348047 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27691810-53a8-4ce2-ad6b-976404854475 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:21.364884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27691810-53a8-4ce2-ad6b-976404854475 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:21.367037 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27691810-53a8-4ce2-ad6b-976404854475 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:21.381297 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1221/2442] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:01:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:01:21.394209 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf0622-5dfe-488f-9521-28a5467ba5c8 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:21.394429 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf0622-5dfe-488f-9521-28a5467ba5c8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:21.394551 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf0622-5dfe-488f-9521-28a5467ba5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:21.425482 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-bddf0622-5dfe-488f-9521-28a5467ba5c8 None None] MFA Rules not processed for user `581ae6d40d34447da6651f3b2c21bdf0`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:01:21.426796 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bddf0622-5dfe-488f-9521-28a5467ba5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:21.487864 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1222/2443] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:01:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:01:21.527231 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27691810-53a8-4ce2-ad6b-976404854475 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:21.528729 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-27691810-53a8-4ce2-ad6b-976404854475 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:21.623618 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27691810-53a8-4ce2-ad6b-976404854475 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:21.628230 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdaa699-ff0d-46c5-ace3-3a93fab7b32d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98560723b49940d38a6a5e9cf3f5d3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:21.628891 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdaa699-ff0d-46c5-ace3-3a93fab7b32d service glance] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:21.629183 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdaa699-ff0d-46c5-ace3-3a93fab7b32d service glance] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:21.629465 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdaa699-ff0d-46c5-ace3-3a93fab7b32d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:21.631378 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0fdaa699-ff0d-46c5-ace3-3a93fab7b32d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:21.673964 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddf0622-5dfe-488f-9521-28a5467ba5c8 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:21.675626 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bddf0622-5dfe-488f-9521-28a5467ba5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:21.684172 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:01:21.692440 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddf0622-5dfe-488f-9521-28a5467ba5c8 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:21.697715 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff388d5-41a1-4118-9be9-58acfe9f3080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:21.699656 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff388d5-41a1-4118-9be9-58acfe9f3080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:21.699871 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff388d5-41a1-4118-9be9-58acfe9f3080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:21.700099 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff388d5-41a1-4118-9be9-58acfe9f3080 admin admin] PATH_INFO: `/v3/projects/b27b2c7d706c4999bdb520e68efb178b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:21.702342 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff388d5-41a1-4118-9be9-58acfe9f3080 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b27b2c7d706c4999bdb520e68efb178b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:21.705196 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff388d5-41a1-4118-9be9-58acfe9f3080 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:21.723329 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdaa699-ff0d-46c5-ace3-3a93fab7b32d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:21.724838 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdaa699-ff0d-46c5-ace3-3a93fab7b32d service glance] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:21.737622 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1222/2444] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:01:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:01:21.759880 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-aff388d5-41a1-4118-9be9-58acfe9f3080 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b27b2c7d706c4999bdb520e68efb178b', 'request_id': 'req-aff388d5-41a1-4118-9be9-58acfe9f3080'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:21.769767 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-aff388d5-41a1-4118-9be9-58acfe9f3080 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b27b2c7d706c4999bdb520e68efb178b', 'request_id': 'req-aff388d5-41a1-4118-9be9-58acfe9f3080'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:21.776675 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-aff388d5-41a1-4118-9be9-58acfe9f3080 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b27b2c7d706c4999bdb520e68efb178b', 'request_id': 'req-aff388d5-41a1-4118-9be9-58acfe9f3080'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:21.788875 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-aff388d5-41a1-4118-9be9-58acfe9f3080 admin admin] The token cache is being invalidate because project b27b2c7d706c4999bdb520e68efb178b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:01:21.789158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-aff388d5-41a1-4118-9be9-58acfe9f3080 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:21.789872 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-aff388d5-41a1-4118-9be9-58acfe9f3080 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:21.791461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1223/2445] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:01:21 2022] DELETE /identity/v3/projects/b27b2c7d706c4999bdb520e68efb178b => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:01:21.795446 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdaa699-ff0d-46c5-ace3-3a93fab7b32d service glance] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:21.806734 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdaa699-ff0d-46c5-ace3-3a93fab7b32d service glance] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:21.876623 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c6c75e-4bf9-47c5-aaaf-e652f657f71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:21.877974 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c6c75e-4bf9-47c5-aaaf-e652f657f71d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:21.877974 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c6c75e-4bf9-47c5-aaaf-e652f657f71d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:21.878271 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c6c75e-4bf9-47c5-aaaf-e652f657f71d admin admin] PATH_INFO: `/v3/users/74cb656c25824d99ab50619b9666dde6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:21.879378 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c6c75e-4bf9-47c5-aaaf-e652f657f71d admin admin] RBAC: Authorizing `identity:delete_user(user_id=74cb656c25824d99ab50619b9666dde6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:21.881447 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c6c75e-4bf9-47c5-aaaf-e652f657f71d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:21.937154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-76c6c75e-4bf9-47c5-aaaf-e652f657f71d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74cb656c25824d99ab50619b9666dde6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:21.946290 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-76c6c75e-4bf9-47c5-aaaf-e652f657f71d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74cb656c25824d99ab50619b9666dde6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:21.957557 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-76c6c75e-4bf9-47c5-aaaf-e652f657f71d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74cb656c25824d99ab50619b9666dde6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:21.958320 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-76c6c75e-4bf9-47c5-aaaf-e652f657f71d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74cb656c25824d99ab50619b9666dde6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:21.958899 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-76c6c75e-4bf9-47c5-aaaf-e652f657f71d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74cb656c25824d99ab50619b9666dde6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:21.979742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1223/2446] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:01:21 2022] DELETE /identity/v3/users/74cb656c25824d99ab50619b9666dde6 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:01:22.063410 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff388d5-41a1-4118-9be9-58acfe9f3080 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:22.064910 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-aff388d5-41a1-4118-9be9-58acfe9f3080 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:22.124373 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:01:22.152894 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff388d5-41a1-4118-9be9-58acfe9f3080 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:22.231665 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a0216-4940-4fe1-a7f9-4e742e9f1e60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:22.232910 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a0216-4940-4fe1-a7f9-4e742e9f1e60 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:22.233200 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a0216-4940-4fe1-a7f9-4e742e9f1e60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:22.233654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a0216-4940-4fe1-a7f9-4e742e9f1e60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:22.238603 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9a1a0216-4940-4fe1-a7f9-4e742e9f1e60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:22.254672 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a0216-4940-4fe1-a7f9-4e742e9f1e60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:22.257027 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a0216-4940-4fe1-a7f9-4e742e9f1e60 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:22.283794 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1224/2447] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:01:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:01:22.384984 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c6c75e-4bf9-47c5-aaaf-e652f657f71d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:22.386575 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-76c6c75e-4bf9-47c5-aaaf-e652f657f71d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:22.455539 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:01:22.472507 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c6c75e-4bf9-47c5-aaaf-e652f657f71d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:22.477780 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c9d99-445e-4cba-b1cb-6cbe7a87c136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:22.478610 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c9d99-445e-4cba-b1cb-6cbe7a87c136 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:22.478939 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c9d99-445e-4cba-b1cb-6cbe7a87c136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:22.479161 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c9d99-445e-4cba-b1cb-6cbe7a87c136 admin admin] PATH_INFO: `/v3/projects/ccb031845f8243e895f833badeba8464` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:22.480519 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c9d99-445e-4cba-b1cb-6cbe7a87c136 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccb031845f8243e895f833badeba8464)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:22.482482 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c9d99-445e-4cba-b1cb-6cbe7a87c136 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:22.535619 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-867c9d99-445e-4cba-b1cb-6cbe7a87c136 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccb031845f8243e895f833badeba8464', 'request_id': 'req-867c9d99-445e-4cba-b1cb-6cbe7a87c136'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:22.542154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-867c9d99-445e-4cba-b1cb-6cbe7a87c136 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccb031845f8243e895f833badeba8464', 'request_id': 'req-867c9d99-445e-4cba-b1cb-6cbe7a87c136'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:22.549706 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-867c9d99-445e-4cba-b1cb-6cbe7a87c136 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccb031845f8243e895f833badeba8464', 'request_id': 'req-867c9d99-445e-4cba-b1cb-6cbe7a87c136'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:22.565482 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-867c9d99-445e-4cba-b1cb-6cbe7a87c136 admin admin] The token cache is being invalidate because project ccb031845f8243e895f833badeba8464 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:01:22.565779 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-867c9d99-445e-4cba-b1cb-6cbe7a87c136 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:22.566416 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-867c9d99-445e-4cba-b1cb-6cbe7a87c136 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:22.568017 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1224/2448] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:01:22 2022] DELETE /identity/v3/projects/ccb031845f8243e895f833badeba8464 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:01:22.573522 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60dd89e-f581-4b59-9802-d8329059fa98 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:22.573822 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60dd89e-f581-4b59-9802-d8329059fa98 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:22.574103 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60dd89e-f581-4b59-9802-d8329059fa98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:22.595680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f60dd89e-f581-4b59-9802-d8329059fa98 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:01:22.596939 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f60dd89e-f581-4b59-9802-d8329059fa98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:22.687579 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1225/2449] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:01:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:01:22.690722 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04037e-22ad-43b3-b8ce-a2b8ff1dc46c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:22.691016 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04037e-22ad-43b3-b8ce-a2b8ff1dc46c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:22.691303 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04037e-22ad-43b3-b8ce-a2b8ff1dc46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:22.714040 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8f04037e-22ad-43b3-b8ce-a2b8ff1dc46c None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:01:22.715450 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8f04037e-22ad-43b3-b8ce-a2b8ff1dc46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:22.734901 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1225/2450] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:01:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:01:22.735272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60dd89e-f581-4b59-9802-d8329059fa98 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:22.741742 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60dd89e-f581-4b59-9802-d8329059fa98 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:22.745742 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be14d19-9b22-45eb-bbfc-f9af193754a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:22.746419 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be14d19-9b22-45eb-bbfc-f9af193754a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:22.746709 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be14d19-9b22-45eb-bbfc-f9af193754a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:22.746993 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be14d19-9b22-45eb-bbfc-f9af193754a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:22.747801 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be14d19-9b22-45eb-bbfc-f9af193754a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 20:01:22.748069 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be14d19-9b22-45eb-bbfc-f9af193754a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:22.749776 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be14d19-9b22-45eb-bbfc-f9af193754a7 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:22.763641 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1226/2451] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 20:01:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 20:01:22.769940 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04037e-22ad-43b3-b8ce-a2b8ff1dc46c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:22.779908 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04037e-22ad-43b3-b8ce-a2b8ff1dc46c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:22.786951 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b05a97-4297-4d03-87b2-46e0b6fef546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:22.787867 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b05a97-4297-4d03-87b2-46e0b6fef546 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:22.788270 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b05a97-4297-4d03-87b2-46e0b6fef546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:22.788664 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b05a97-4297-4d03-87b2-46e0b6fef546 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:22.789570 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b05a97-4297-4d03-87b2-46e0b6fef546 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:22.791865 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b05a97-4297-4d03-87b2-46e0b6fef546 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:22.837445 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1226/2452] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:01:22 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:01:22.838611 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be14d19-9b22-45eb-bbfc-f9af193754a7 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:22.849895 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be14d19-9b22-45eb-bbfc-f9af193754a7 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:22.916529 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e3075-ebf6-4b81-ac3e-80199d6a7a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:22.917129 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e3075-ebf6-4b81-ac3e-80199d6a7a3b admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:22.917423 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e3075-ebf6-4b81-ac3e-80199d6a7a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:22.917710 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e3075-ebf6-4b81-ac3e-80199d6a7a3b admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:22.918439 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4e3075-ebf6-4b81-ac3e-80199d6a7a3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:22.920448 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4e3075-ebf6-4b81-ac3e-80199d6a7a3b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:22.953445 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1227/2453] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:01:22 2022] POST /identity/v3/users => generated 287 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:01:22.957470 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b05a97-4297-4d03-87b2-46e0b6fef546 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:22.969339 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b05a97-4297-4d03-87b2-46e0b6fef546 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:22.974815 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c846d34-2f63-40dc-95a9-5d6c6ac4e29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:22.975539 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c846d34-2f63-40dc-95a9-5d6c6ac4e29e admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:22.975843 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c846d34-2f63-40dc-95a9-5d6c6ac4e29e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:22.976118 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c846d34-2f63-40dc-95a9-5d6c6ac4e29e admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:22.976879 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c846d34-2f63-40dc-95a9-5d6c6ac4e29e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:22.978496 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c846d34-2f63-40dc-95a9-5d6c6ac4e29e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:22.995826 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1227/2454] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:01:22.997105 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e3075-ebf6-4b81-ac3e-80199d6a7a3b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:23.008199 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e3075-ebf6-4b81-ac3e-80199d6a7a3b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:23.013017 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54a0cc-d2cc-4731-8b48-eb51a1ecfa1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:23.013689 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54a0cc-d2cc-4731-8b48-eb51a1ecfa1b admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:23.013975 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54a0cc-d2cc-4731-8b48-eb51a1ecfa1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:23.014256 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54a0cc-d2cc-4731-8b48-eb51a1ecfa1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:23.014998 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec54a0cc-d2cc-4731-8b48-eb51a1ecfa1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:23.017964 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec54a0cc-d2cc-4731-8b48-eb51a1ecfa1b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:23.032304 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1228/2455] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:01:23.033626 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c846d34-2f63-40dc-95a9-5d6c6ac4e29e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:23.047998 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c846d34-2f63-40dc-95a9-5d6c6ac4e29e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:23.054712 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751fbf49-8b29-4d09-b218-9a864fb2514e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:23.055923 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fbf49-8b29-4d09-b218-9a864fb2514e admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:23.056323 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fbf49-8b29-4d09-b218-9a864fb2514e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:23.056700 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fbf49-8b29-4d09-b218-9a864fb2514e admin admin] PATH_INFO: `/v3/projects/b369a0bf07c5405dac8b37a8e947fbc7/users/d5cb3fd3c84146e4ade535d949d2ea10/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:23.078048 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751fbf49-8b29-4d09-b218-9a864fb2514e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b369a0bf07c5405dac8b37a8e947fbc7, user_id=d5cb3fd3c84146e4ade535d949d2ea10, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:23.081682 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751fbf49-8b29-4d09-b218-9a864fb2514e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:23.097466 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1228/2456] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:01:23 2022] PUT /identity/v3/projects/b369a0bf07c5405dac8b37a8e947fbc7/users/d5cb3fd3c84146e4ade535d949d2ea10/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:01:23.109146 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba16fc-c38d-45ec-bfac-4d001fb8f1da None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:23.110079 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba16fc-c38d-45ec-bfac-4d001fb8f1da None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:23.110368 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba16fc-c38d-45ec-bfac-4d001fb8f1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:23.119280 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:01:23.146654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-97ba16fc-c38d-45ec-bfac-4d001fb8f1da None None] MFA Rules not processed for user `d5cb3fd3c84146e4ade535d949d2ea10`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:01:23.147940 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-97ba16fc-c38d-45ec-bfac-4d001fb8f1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:23.224365 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1229/2457] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:01:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:01:23.236584 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751fbf49-8b29-4d09-b218-9a864fb2514e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:23.238413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-751fbf49-8b29-4d09-b218-9a864fb2514e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:23.310687 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751fbf49-8b29-4d09-b218-9a864fb2514e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:23.315649 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcabb8a-d3f5-496d-8a3f-81d790aa351f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:23.316501 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcabb8a-d3f5-496d-8a3f-81d790aa351f service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:23.316733 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcabb8a-d3f5-496d-8a3f-81d790aa351f service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:23.316865 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcabb8a-d3f5-496d-8a3f-81d790aa351f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:23.328896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcabb8a-d3f5-496d-8a3f-81d790aa351f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:23.330282 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcabb8a-d3f5-496d-8a3f-81d790aa351f service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:23.351404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1229/2458] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:01:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:01:23.381095 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba16fc-c38d-45ec-bfac-4d001fb8f1da None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:23.471776 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba16fc-c38d-45ec-bfac-4d001fb8f1da None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:23.476449 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a51066-f549-4133-8a6b-3ac59f9c0ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:23.478636 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a51066-f549-4133-8a6b-3ac59f9c0ef1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:23.478636 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a51066-f549-4133-8a6b-3ac59f9c0ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:23.478636 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a51066-f549-4133-8a6b-3ac59f9c0ef1 admin admin] PATH_INFO: `/v3/users/d5cb3fd3c84146e4ade535d949d2ea10` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:23.479174 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a51066-f549-4133-8a6b-3ac59f9c0ef1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5cb3fd3c84146e4ade535d949d2ea10)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:23.481186 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a51066-f549-4133-8a6b-3ac59f9c0ef1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:23.537686 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-87a51066-f549-4133-8a6b-3ac59f9c0ef1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5cb3fd3c84146e4ade535d949d2ea10'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:23.546748 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-87a51066-f549-4133-8a6b-3ac59f9c0ef1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5cb3fd3c84146e4ade535d949d2ea10'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:23.554872 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-87a51066-f549-4133-8a6b-3ac59f9c0ef1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5cb3fd3c84146e4ade535d949d2ea10'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:23.555761 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-87a51066-f549-4133-8a6b-3ac59f9c0ef1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5cb3fd3c84146e4ade535d949d2ea10'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:23.574154 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-87a51066-f549-4133-8a6b-3ac59f9c0ef1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5cb3fd3c84146e4ade535d949d2ea10'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:23.587378 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1230/2459] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:01:23 2022] DELETE /identity/v3/users/d5cb3fd3c84146e4ade535d949d2ea10 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:01:23.592386 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b4685-f5b5-42e2-a8e3-68db29c73b3f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:23.593144 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b4685-f5b5-42e2-a8e3-68db29c73b3f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:23.593144 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b4685-f5b5-42e2-a8e3-68db29c73b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:23.613627 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-753b4685-f5b5-42e2-a8e3-68db29c73b3f None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:01:23.614900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-753b4685-f5b5-42e2-a8e3-68db29c73b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:23.637355 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:01:23.728197 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1230/2460] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:01:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:01:23.742613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a51066-f549-4133-8a6b-3ac59f9c0ef1 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:23.744730 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-87a51066-f549-4133-8a6b-3ac59f9c0ef1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:23.770323 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:01:23.832645 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a51066-f549-4133-8a6b-3ac59f9c0ef1 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:23.836801 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ebc22-be41-4eda-bd75-12ec7363868c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:23.837470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ebc22-be41-4eda-bd75-12ec7363868c service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:23.837759 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ebc22-be41-4eda-bd75-12ec7363868c service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:23.838049 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ebc22-be41-4eda-bd75-12ec7363868c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:23.850387 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36ebc22-be41-4eda-bd75-12ec7363868c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:23.852113 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36ebc22-be41-4eda-bd75-12ec7363868c service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:23.863732 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1231/2461] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:01:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:01:24.205590 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b4685-f5b5-42e2-a8e3-68db29c73b3f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:24.207317 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-753b4685-f5b5-42e2-a8e3-68db29c73b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:24.226710 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b4685-f5b5-42e2-a8e3-68db29c73b3f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:24.240984 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6feb95-e62d-40e4-a67b-7e35f5af7b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:24.241872 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6feb95-e62d-40e4-a67b-7e35f5af7b50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:24.242166 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6feb95-e62d-40e4-a67b-7e35f5af7b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:24.242436 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6feb95-e62d-40e4-a67b-7e35f5af7b50 admin admin] PATH_INFO: `/v3/projects/b369a0bf07c5405dac8b37a8e947fbc7` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:24.243800 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6feb95-e62d-40e4-a67b-7e35f5af7b50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b369a0bf07c5405dac8b37a8e947fbc7)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:24.245965 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6feb95-e62d-40e4-a67b-7e35f5af7b50 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:24.326034 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ad6feb95-e62d-40e4-a67b-7e35f5af7b50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b369a0bf07c5405dac8b37a8e947fbc7', 'request_id': 'req-ad6feb95-e62d-40e4-a67b-7e35f5af7b50'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:24.334438 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ad6feb95-e62d-40e4-a67b-7e35f5af7b50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b369a0bf07c5405dac8b37a8e947fbc7', 'request_id': 'req-ad6feb95-e62d-40e4-a67b-7e35f5af7b50'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:24.342441 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ad6feb95-e62d-40e4-a67b-7e35f5af7b50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b369a0bf07c5405dac8b37a8e947fbc7', 'request_id': 'req-ad6feb95-e62d-40e4-a67b-7e35f5af7b50'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:24.357976 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ad6feb95-e62d-40e4-a67b-7e35f5af7b50 admin admin] The token cache is being invalidate because project b369a0bf07c5405dac8b37a8e947fbc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:01:24.358270 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ad6feb95-e62d-40e4-a67b-7e35f5af7b50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:24.359013 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ad6feb95-e62d-40e4-a67b-7e35f5af7b50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:24.360821 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1231/2462] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:01:24 2022] DELETE /identity/v3/projects/b369a0bf07c5405dac8b37a8e947fbc7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:01:24.367187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea54da-7b88-46d3-8659-3958fd7e0431 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:24.367785 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea54da-7b88-46d3-8659-3958fd7e0431 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:24.368160 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea54da-7b88-46d3-8659-3958fd7e0431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:24.399346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-acea54da-7b88-46d3-8659-3958fd7e0431 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:01:24.400632 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-acea54da-7b88-46d3-8659-3958fd7e0431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:24.483419 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1232/2463] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:01:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 20:01:24.492455 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa5e86-12f6-4497-9166-9e43631dc949 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:24.492757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa5e86-12f6-4497-9166-9e43631dc949 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:24.493047 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa5e86-12f6-4497-9166-9e43631dc949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:24.522945 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5bfa5e86-12f6-4497-9166-9e43631dc949 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:01:24.524222 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5bfa5e86-12f6-4497-9166-9e43631dc949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:24.532463 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1232/2464] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:01:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 20:01:24.536835 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea54da-7b88-46d3-8659-3958fd7e0431 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:24.547461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea54da-7b88-46d3-8659-3958fd7e0431 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:24.552448 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eb1b50-0968-4ba3-bc31-f058bd0af63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:24.553140 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb1b50-0968-4ba3-bc31-f058bd0af63f admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:24.553430 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb1b50-0968-4ba3-bc31-f058bd0af63f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:24.553712 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb1b50-0968-4ba3-bc31-f058bd0af63f admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:24.554462 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eb1b50-0968-4ba3-bc31-f058bd0af63f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 20:01:24.554829 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eb1b50-0968-4ba3-bc31-f058bd0af63f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:24.556509 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eb1b50-0968-4ba3-bc31-f058bd0af63f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:24.574141 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1233/2465] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 20:01:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 20:01:24.578967 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa5e86-12f6-4497-9166-9e43631dc949 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:24.591140 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa5e86-12f6-4497-9166-9e43631dc949 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:24.597650 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb18913-9b62-46c4-812a-c76a6a9d3600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:24.598466 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb18913-9b62-46c4-812a-c76a6a9d3600 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:24.598879 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb18913-9b62-46c4-812a-c76a6a9d3600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:24.599261 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb18913-9b62-46c4-812a-c76a6a9d3600 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:24.600084 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb18913-9b62-46c4-812a-c76a6a9d3600 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:24.602115 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb18913-9b62-46c4-812a-c76a6a9d3600 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:24.635876 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1233/2466] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:01:24 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 20:01:24.640058 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eb1b50-0968-4ba3-bc31-f058bd0af63f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:24.656694 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eb1b50-0968-4ba3-bc31-f058bd0af63f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:24.732116 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd8de7-2eb7-4480-9a2c-0a46a7bfa900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:24.733356 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd8de7-2eb7-4480-9a2c-0a46a7bfa900 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:24.733356 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd8de7-2eb7-4480-9a2c-0a46a7bfa900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:24.733655 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd8de7-2eb7-4480-9a2c-0a46a7bfa900 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:24.734428 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd8de7-2eb7-4480-9a2c-0a46a7bfa900 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:24.736650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd8de7-2eb7-4480-9a2c-0a46a7bfa900 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:24.759972 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1234/2467] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:01:24 2022] POST /identity/v3/users => generated 288 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:01:24.764483 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb18913-9b62-46c4-812a-c76a6a9d3600 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:24.777847 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb18913-9b62-46c4-812a-c76a6a9d3600 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:24.783960 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e95c66d-fa51-4bc6-a7a1-f5e95ce0077b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:24.784994 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95c66d-fa51-4bc6-a7a1-f5e95ce0077b admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:24.785571 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95c66d-fa51-4bc6-a7a1-f5e95ce0077b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:24.786144 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95c66d-fa51-4bc6-a7a1-f5e95ce0077b admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:24.787291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e95c66d-fa51-4bc6-a7a1-f5e95ce0077b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:24.789158 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e95c66d-fa51-4bc6-a7a1-f5e95ce0077b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:24.802457 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1234/2468] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:01:24.806998 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd8de7-2eb7-4480-9a2c-0a46a7bfa900 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:24.830503 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd8de7-2eb7-4480-9a2c-0a46a7bfa900 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:24.836120 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19eea2d-dbf3-4128-a188-277a68432a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:24.836809 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19eea2d-dbf3-4128-a188-277a68432a42 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:24.837153 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19eea2d-dbf3-4128-a188-277a68432a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:24.837439 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19eea2d-dbf3-4128-a188-277a68432a42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:24.838188 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19eea2d-dbf3-4128-a188-277a68432a42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:24.840821 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19eea2d-dbf3-4128-a188-277a68432a42 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:24.871910 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1235/2469] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:01:24.880390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e95c66d-fa51-4bc6-a7a1-f5e95ce0077b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:24.903681 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e95c66d-fa51-4bc6-a7a1-f5e95ce0077b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:24.920948 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d993c7e1-2501-47c8-ad3c-d22e43c2c3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:24.924564 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993c7e1-2501-47c8-ad3c-d22e43c2c3be admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:24.925389 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993c7e1-2501-47c8-ad3c-d22e43c2c3be admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:24.925949 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993c7e1-2501-47c8-ad3c-d22e43c2c3be admin admin] PATH_INFO: `/v3/projects/e6261e7ddf494aaa966967ddb5a1fd44/users/08bbae0210ba41a19a51a2fbc731c9f9/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:24.959285 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d993c7e1-2501-47c8-ad3c-d22e43c2c3be admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6261e7ddf494aaa966967ddb5a1fd44, user_id=08bbae0210ba41a19a51a2fbc731c9f9, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:24.962890 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d993c7e1-2501-47c8-ad3c-d22e43c2c3be admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:24.992039 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1235/2470] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:01:24 2022] PUT /identity/v3/projects/e6261e7ddf494aaa966967ddb5a1fd44/users/08bbae0210ba41a19a51a2fbc731c9f9/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:01:25.003295 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212b79d-bada-4f8d-9db5-760fa53724d4 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:25.003622 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212b79d-bada-4f8d-9db5-760fa53724d4 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:25.003904 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212b79d-bada-4f8d-9db5-760fa53724d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:25.036332 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7212b79d-bada-4f8d-9db5-760fa53724d4 None None] MFA Rules not processed for user `08bbae0210ba41a19a51a2fbc731c9f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:01:25.038283 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7212b79d-bada-4f8d-9db5-760fa53724d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:25.053828 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:01:25.136136 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1236/2471] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:01:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:01:25.141066 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d993c7e1-2501-47c8-ad3c-d22e43c2c3be admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:25.191260 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:01:25.211003 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d993c7e1-2501-47c8-ad3c-d22e43c2c3be admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:25.215804 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294633c-7ff5-4378-a558-b533d424cbe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:25.216473 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294633c-7ff5-4378-a558-b533d424cbe0 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:25.216762 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294633c-7ff5-4378-a558-b533d424cbe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:25.217042 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294633c-7ff5-4378-a558-b533d424cbe0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:25.217750 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6294633c-7ff5-4378-a558-b533d424cbe0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:25.220433 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6294633c-7ff5-4378-a558-b533d424cbe0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:25.253508 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1236/2472] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:01:25 2022] POST /identity/v3/projects => generated 375 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 20:01:25.257529 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7212b79d-bada-4f8d-9db5-760fa53724d4 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:25.270159 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7212b79d-bada-4f8d-9db5-760fa53724d4 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:25.338423 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496c965-7425-4bd1-a0fc-662867c3f6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:25.339118 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496c965-7425-4bd1-a0fc-662867c3f6b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:25.339406 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496c965-7425-4bd1-a0fc-662867c3f6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:25.339715 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496c965-7425-4bd1-a0fc-662867c3f6b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:25.340438 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c496c965-7425-4bd1-a0fc-662867c3f6b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:25.342426 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c496c965-7425-4bd1-a0fc-662867c3f6b5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:25.369425 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1237/2473] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:01:25 2022] POST /identity/v3/users => generated 288 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:01:25.373843 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294633c-7ff5-4378-a558-b533d424cbe0 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:25.385276 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294633c-7ff5-4378-a558-b533d424cbe0 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:25.390394 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d9746-f525-4aef-af30-1c0ce1769795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:25.391666 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d9746-f525-4aef-af30-1c0ce1769795 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:25.391979 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d9746-f525-4aef-af30-1c0ce1769795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:25.392262 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d9746-f525-4aef-af30-1c0ce1769795 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:25.393117 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0d9746-f525-4aef-af30-1c0ce1769795 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:25.394756 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0d9746-f525-4aef-af30-1c0ce1769795 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:25.406685 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1237/2474] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:01:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:01:25.411307 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496c965-7425-4bd1-a0fc-662867c3f6b5 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:25.421637 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496c965-7425-4bd1-a0fc-662867c3f6b5 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:25.427578 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc29f9-1e8c-4e13-9263-0aabc2f51cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:25.428255 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc29f9-1e8c-4e13-9263-0aabc2f51cec admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:25.428541 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc29f9-1e8c-4e13-9263-0aabc2f51cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:25.428820 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc29f9-1e8c-4e13-9263-0aabc2f51cec admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:25.429551 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effc29f9-1e8c-4e13-9263-0aabc2f51cec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:25.431176 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effc29f9-1e8c-4e13-9263-0aabc2f51cec admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:25.442196 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1238/2475] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:01:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:01:25.446477 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d9746-f525-4aef-af30-1c0ce1769795 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:25.457993 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d9746-f525-4aef-af30-1c0ce1769795 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:25.462802 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc0a6ea-f0a0-42c0-ad85-3b6e6e8c6044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:25.463738 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0a6ea-f0a0-42c0-ad85-3b6e6e8c6044 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:25.464025 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0a6ea-f0a0-42c0-ad85-3b6e6e8c6044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:25.464315 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0a6ea-f0a0-42c0-ad85-3b6e6e8c6044 admin admin] PATH_INFO: `/v3/projects/3d0508b6d5e7459293e0897e3344a8e3/users/b2ec4b391ac7417394201a1a585ad775/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:25.480876 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc0a6ea-f0a0-42c0-ad85-3b6e6e8c6044 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d0508b6d5e7459293e0897e3344a8e3, user_id=b2ec4b391ac7417394201a1a585ad775, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:25.483792 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc0a6ea-f0a0-42c0-ad85-3b6e6e8c6044 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:25.502182 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1238/2476] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:01:25 2022] PUT /identity/v3/projects/3d0508b6d5e7459293e0897e3344a8e3/users/b2ec4b391ac7417394201a1a585ad775/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:01:25.512556 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411e2b1-f019-4b32-965e-52b811995328 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:25.512865 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411e2b1-f019-4b32-965e-52b811995328 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:25.513127 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411e2b1-f019-4b32-965e-52b811995328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:25.570215 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b411e2b1-f019-4b32-965e-52b811995328 None None] MFA Rules not processed for user `b2ec4b391ac7417394201a1a585ad775`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:01:25.572888 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b411e2b1-f019-4b32-965e-52b811995328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:25.665137 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1239/2477] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:01:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:01:25.669819 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc0a6ea-f0a0-42c0-ad85-3b6e6e8c6044 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:25.767723 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc0a6ea-f0a0-42c0-ad85-3b6e6e8c6044 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:25.775207 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5949c222-db8b-4a4e-bbdd-db2d88bff3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:25.776139 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949c222-db8b-4a4e-bbdd-db2d88bff3b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:25.776406 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949c222-db8b-4a4e-bbdd-db2d88bff3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:25.776794 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5949c222-db8b-4a4e-bbdd-db2d88bff3b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:25.777925 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5949c222-db8b-4a4e-bbdd-db2d88bff3b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:25.780240 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5949c222-db8b-4a4e-bbdd-db2d88bff3b9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:25.822825 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1239/2478] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:01:25 2022] POST /identity/v3/projects => generated 377 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:01:25.828687 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b411e2b1-f019-4b32-965e-52b811995328 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:25.843449 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b411e2b1-f019-4b32-965e-52b811995328 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:25.865113 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:01:25.936575 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beda1fa-b3dd-4fe2-9a8f-c82a72f1ebfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:25.938374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beda1fa-b3dd-4fe2-9a8f-c82a72f1ebfa admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:25.938738 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beda1fa-b3dd-4fe2-9a8f-c82a72f1ebfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:25.939171 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beda1fa-b3dd-4fe2-9a8f-c82a72f1ebfa admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:25.940219 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beda1fa-b3dd-4fe2-9a8f-c82a72f1ebfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:25.942697 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beda1fa-b3dd-4fe2-9a8f-c82a72f1ebfa admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:25.976291 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1240/2479] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:01:25 2022] POST /identity/v3/users => generated 295 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:01:25.977966 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5949c222-db8b-4a4e-bbdd-db2d88bff3b9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:25.990182 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5949c222-db8b-4a4e-bbdd-db2d88bff3b9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:25.995479 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5fef16-a91f-4ea9-aef8-d338e93b0023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:25.996788 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5fef16-a91f-4ea9-aef8-d338e93b0023 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:25.996788 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5fef16-a91f-4ea9-aef8-d338e93b0023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:25.997093 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5fef16-a91f-4ea9-aef8-d338e93b0023 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:25.997809 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5fef16-a91f-4ea9-aef8-d338e93b0023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:25.999697 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5fef16-a91f-4ea9-aef8-d338e93b0023 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:26.020358 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1240/2480] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:01:25 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:01:26.020904 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beda1fa-b3dd-4fe2-9a8f-c82a72f1ebfa admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:26.042744 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beda1fa-b3dd-4fe2-9a8f-c82a72f1ebfa admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:26.057256 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9d337-3000-45e1-a439-3896773a6e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:26.060699 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9d337-3000-45e1-a439-3896773a6e51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:26.061166 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9d337-3000-45e1-a439-3896773a6e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:26.063298 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9d337-3000-45e1-a439-3896773a6e51 admin admin] PATH_INFO: `/v3/projects/a071c8f2530d4bcea4efa92b6c31ea57/users/89970b8043c54cb6986482e917787808/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:26.096964 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a9d337-3000-45e1-a439-3896773a6e51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a071c8f2530d4bcea4efa92b6c31ea57, user_id=89970b8043c54cb6986482e917787808, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:26.112633 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a9d337-3000-45e1-a439-3896773a6e51 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:26.154603 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1241/2481] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:01:26 2022] PUT /identity/v3/projects/a071c8f2530d4bcea4efa92b6c31ea57/users/89970b8043c54cb6986482e917787808/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:01:26.155948 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5fef16-a91f-4ea9-aef8-d338e93b0023 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:26.178342 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5fef16-a91f-4ea9-aef8-d338e93b0023 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:26.282138 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4debe66-084b-45b1-a511-6394ccecde82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:26.282734 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4debe66-084b-45b1-a511-6394ccecde82 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:26.283065 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4debe66-084b-45b1-a511-6394ccecde82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:26.285289 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4debe66-084b-45b1-a511-6394ccecde82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:26.286717 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4debe66-084b-45b1-a511-6394ccecde82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:26.290440 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4debe66-084b-45b1-a511-6394ccecde82 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:26.308515 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1241/2482] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:01:26.314456 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9d337-3000-45e1-a439-3896773a6e51 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:26.329712 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9d337-3000-45e1-a439-3896773a6e51 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:26.336828 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0161316-2434-44a6-9c68-fca4e67d59b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:26.336828 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0161316-2434-44a6-9c68-fca4e67d59b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:26.337193 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0161316-2434-44a6-9c68-fca4e67d59b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:26.337450 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0161316-2434-44a6-9c68-fca4e67d59b7 admin admin] PATH_INFO: `/v3/projects/a071c8f2530d4bcea4efa92b6c31ea57/users/89970b8043c54cb6986482e917787808/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:26.340031 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0161316-2434-44a6-9c68-fca4e67d59b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a071c8f2530d4bcea4efa92b6c31ea57, user_id=89970b8043c54cb6986482e917787808, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:26.343303 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0161316-2434-44a6-9c68-fca4e67d59b7 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:26.376451 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1242/2483] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:01:26 2022] PUT /identity/v3/projects/a071c8f2530d4bcea4efa92b6c31ea57/users/89970b8043c54cb6986482e917787808/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 20:01:26.381252 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b35681-210f-4bf3-b238-2e913c3fd3aa None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:26.381711 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b35681-210f-4bf3-b238-2e913c3fd3aa None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:26.382164 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b35681-210f-4bf3-b238-2e913c3fd3aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:26.391866 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:01:26.428650 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c5b35681-210f-4bf3-b238-2e913c3fd3aa None None] MFA Rules not processed for user `89970b8043c54cb6986482e917787808`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:01:26.429997 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c5b35681-210f-4bf3-b238-2e913c3fd3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:26.523043 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1242/2484] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:01:26 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:01:26.538563 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0161316-2434-44a6-9c68-fca4e67d59b7 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:26.540388 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f0161316-2434-44a6-9c68-fca4e67d59b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:26.629633 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0161316-2434-44a6-9c68-fca4e67d59b7 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:26.637931 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc11535-105c-4ba4-8ef7-ebfcf547ef56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:26.638990 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc11535-105c-4ba4-8ef7-ebfcf547ef56 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:26.640188 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc11535-105c-4ba4-8ef7-ebfcf547ef56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:26.641121 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc11535-105c-4ba4-8ef7-ebfcf547ef56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:26.683104 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b35681-210f-4bf3-b238-2e913c3fd3aa None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:26.684891 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c5b35681-210f-4bf3-b238-2e913c3fd3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:26.723824 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc11535-105c-4ba4-8ef7-ebfcf547ef56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:26.725295 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc11535-105c-4ba4-8ef7-ebfcf547ef56 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:26.737444 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1243/2485] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:01:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:01:26.759646 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b35681-210f-4bf3-b238-2e913c3fd3aa None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:26.766665 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f93e1-52f3-4917-8221-b6e8d8a954a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:26.767898 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f93e1-52f3-4917-8221-b6e8d8a954a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:26.768399 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f93e1-52f3-4917-8221-b6e8d8a954a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:26.768927 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f93e1-52f3-4917-8221-b6e8d8a954a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:26.772103 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5c8f93e1-52f3-4917-8221-b6e8d8a954a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:26.874330 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8f93e1-52f3-4917-8221-b6e8d8a954a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:26.878863 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8f93e1-52f3-4917-8221-b6e8d8a954a0 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:26.904955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1243/2486] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:01:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 20:01:27.831095 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc11535-105c-4ba4-8ef7-ebfcf547ef56 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:27.835948 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7cc11535-105c-4ba4-8ef7-ebfcf547ef56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:27.856874 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc11535-105c-4ba4-8ef7-ebfcf547ef56 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:27.862301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65240f0f-c3b1-41bb-a5ed-30d2f52d4e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:27.863199 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65240f0f-c3b1-41bb-a5ed-30d2f52d4e55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:27.863487 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65240f0f-c3b1-41bb-a5ed-30d2f52d4e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:27.863808 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65240f0f-c3b1-41bb-a5ed-30d2f52d4e55 admin admin] PATH_INFO: `/v3/users/324738dd872e445ba845a101f45fa595` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:27.865265 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65240f0f-c3b1-41bb-a5ed-30d2f52d4e55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=324738dd872e445ba845a101f45fa595)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:27.867736 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65240f0f-c3b1-41bb-a5ed-30d2f52d4e55 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:27.900629 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:01:27.932330 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-65240f0f-c3b1-41bb-a5ed-30d2f52d4e55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '324738dd872e445ba845a101f45fa595'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:27.941472 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-65240f0f-c3b1-41bb-a5ed-30d2f52d4e55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '324738dd872e445ba845a101f45fa595'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:27.950330 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-65240f0f-c3b1-41bb-a5ed-30d2f52d4e55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '324738dd872e445ba845a101f45fa595'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:27.951686 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-65240f0f-c3b1-41bb-a5ed-30d2f52d4e55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '324738dd872e445ba845a101f45fa595'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:27.968369 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-65240f0f-c3b1-41bb-a5ed-30d2f52d4e55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '324738dd872e445ba845a101f45fa595'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:27.975666 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1244/2487] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:01:27 2022] DELETE /identity/v3/users/324738dd872e445ba845a101f45fa595 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:01:28.171092 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f93e1-52f3-4917-8221-b6e8d8a954a0 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:28.184154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f93e1-52f3-4917-8221-b6e8d8a954a0 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:28.225386 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:01:28.252891 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3aad29-c6e3-4d5e-bc05-b4ac3d794a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:28.253777 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3aad29-c6e3-4d5e-bc05-b4ac3d794a12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:28.254064 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3aad29-c6e3-4d5e-bc05-b4ac3d794a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:28.254341 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3aad29-c6e3-4d5e-bc05-b4ac3d794a12 admin admin] PATH_INFO: `/v3/projects/43cfdd0ba7de4e46a17d4c0d698159cf` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:28.255621 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3aad29-c6e3-4d5e-bc05-b4ac3d794a12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43cfdd0ba7de4e46a17d4c0d698159cf)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:28.257551 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3aad29-c6e3-4d5e-bc05-b4ac3d794a12 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:28.311648 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4c3aad29-c6e3-4d5e-bc05-b4ac3d794a12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43cfdd0ba7de4e46a17d4c0d698159cf', 'request_id': 'req-4c3aad29-c6e3-4d5e-bc05-b4ac3d794a12'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:28.320221 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4c3aad29-c6e3-4d5e-bc05-b4ac3d794a12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43cfdd0ba7de4e46a17d4c0d698159cf', 'request_id': 'req-4c3aad29-c6e3-4d5e-bc05-b4ac3d794a12'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:28.327162 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4c3aad29-c6e3-4d5e-bc05-b4ac3d794a12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43cfdd0ba7de4e46a17d4c0d698159cf', 'request_id': 'req-4c3aad29-c6e3-4d5e-bc05-b4ac3d794a12'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:28.342861 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4c3aad29-c6e3-4d5e-bc05-b4ac3d794a12 admin admin] The token cache is being invalidate because project 43cfdd0ba7de4e46a17d4c0d698159cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:01:28.342861 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4c3aad29-c6e3-4d5e-bc05-b4ac3d794a12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:28.343404 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4c3aad29-c6e3-4d5e-bc05-b4ac3d794a12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:28.345907 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1244/2488] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:01:28 2022] DELETE /identity/v3/projects/43cfdd0ba7de4e46a17d4c0d698159cf => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:01:28.588832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65240f0f-c3b1-41bb-a5ed-30d2f52d4e55 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:28.590485 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-65240f0f-c3b1-41bb-a5ed-30d2f52d4e55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:28.662761 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65240f0f-c3b1-41bb-a5ed-30d2f52d4e55 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:28.717217 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef78d9-eed3-407b-b79d-fe01cf5409a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:28.717796 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef78d9-eed3-407b-b79d-fe01cf5409a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:28.718084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef78d9-eed3-407b-b79d-fe01cf5409a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:28.718351 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef78d9-eed3-407b-b79d-fe01cf5409a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:28.721976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-72ef78d9-eed3-407b-b79d-fe01cf5409a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:28.787500 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef78d9-eed3-407b-b79d-fe01cf5409a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:28.788989 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef78d9-eed3-407b-b79d-fe01cf5409a0 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:28.798881 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1245/2489] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:01:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:01:29.496124 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3aad29-c6e3-4d5e-bc05-b4ac3d794a12 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:29.497598 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4c3aad29-c6e3-4d5e-bc05-b4ac3d794a12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:29.510847 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3aad29-c6e3-4d5e-bc05-b4ac3d794a12 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:29.517177 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50df82a-d490-4d8b-8333-54a65c4befd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:29.517972 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50df82a-d490-4d8b-8333-54a65c4befd7 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:29.518717 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50df82a-d490-4d8b-8333-54a65c4befd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:29.519013 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50df82a-d490-4d8b-8333-54a65c4befd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:29.521745 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d50df82a-d490-4d8b-8333-54a65c4befd7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:29.611745 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50df82a-d490-4d8b-8333-54a65c4befd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:29.613348 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50df82a-d490-4d8b-8333-54a65c4befd7 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:29.624482 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1245/2490] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:01:29 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:01:33.771526 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef78d9-eed3-407b-b79d-fe01cf5409a0 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:33.773237 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-72ef78d9-eed3-407b-b79d-fe01cf5409a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:33.791064 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:01:33.868034 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef78d9-eed3-407b-b79d-fe01cf5409a0 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:33.875058 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd0a04a-e3fd-4d4c-a16b-54e680310d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:33.876121 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0a04a-e3fd-4d4c-a16b-54e680310d71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:33.876408 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0a04a-e3fd-4d4c-a16b-54e680310d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:33.876688 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0a04a-e3fd-4d4c-a16b-54e680310d71 admin admin] PATH_INFO: `/v3/users/08bbae0210ba41a19a51a2fbc731c9f9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:33.877974 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd0a04a-e3fd-4d4c-a16b-54e680310d71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08bbae0210ba41a19a51a2fbc731c9f9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:33.879988 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd0a04a-e3fd-4d4c-a16b-54e680310d71 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:33.943301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ecd0a04a-e3fd-4d4c-a16b-54e680310d71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08bbae0210ba41a19a51a2fbc731c9f9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:33.950107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ecd0a04a-e3fd-4d4c-a16b-54e680310d71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08bbae0210ba41a19a51a2fbc731c9f9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:33.960163 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ecd0a04a-e3fd-4d4c-a16b-54e680310d71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08bbae0210ba41a19a51a2fbc731c9f9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:33.960919 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ecd0a04a-e3fd-4d4c-a16b-54e680310d71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08bbae0210ba41a19a51a2fbc731c9f9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:33.974900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ecd0a04a-e3fd-4d4c-a16b-54e680310d71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08bbae0210ba41a19a51a2fbc731c9f9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:33.981367 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1246/2491] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:01:33 2022] DELETE /identity/v3/users/08bbae0210ba41a19a51a2fbc731c9f9 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:01:33.988900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec789df-f0d3-4a3a-b72f-d0c1edab265c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:33.988900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec789df-f0d3-4a3a-b72f-d0c1edab265c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:33.990073 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec789df-f0d3-4a3a-b72f-d0c1edab265c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:34.012214 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2ec789df-f0d3-4a3a-b72f-d0c1edab265c None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:01:34.013482 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2ec789df-f0d3-4a3a-b72f-d0c1edab265c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:34.108094 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1246/2492] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:01:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:01:34.120257 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd0a04a-e3fd-4d4c-a16b-54e680310d71 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:34.121858 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ecd0a04a-e3fd-4d4c-a16b-54e680310d71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:34.202089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd0a04a-e3fd-4d4c-a16b-54e680310d71 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:34.206853 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d022aa1-ccc4-49a2-8c40-e2d7665532ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:34.207570 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d022aa1-ccc4-49a2-8c40-e2d7665532ad service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:34.207866 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d022aa1-ccc4-49a2-8c40-e2d7665532ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:34.208146 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d022aa1-ccc4-49a2-8c40-e2d7665532ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:34.219480 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d022aa1-ccc4-49a2-8c40-e2d7665532ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:34.221141 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d022aa1-ccc4-49a2-8c40-e2d7665532ad service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:34.233960 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1247/2493] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:01:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:01:34.654178 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec789df-f0d3-4a3a-b72f-d0c1edab265c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:34.656934 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2ec789df-f0d3-4a3a-b72f-d0c1edab265c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:34.665438 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:01:34.674664 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec789df-f0d3-4a3a-b72f-d0c1edab265c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:34.684924 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ded053d-3088-408c-a01b-2df3c5b4ce13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:34.686825 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded053d-3088-408c-a01b-2df3c5b4ce13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:34.687448 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded053d-3088-408c-a01b-2df3c5b4ce13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:34.687917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded053d-3088-408c-a01b-2df3c5b4ce13 admin admin] PATH_INFO: `/v3/projects/e6261e7ddf494aaa966967ddb5a1fd44` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:34.689627 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ded053d-3088-408c-a01b-2df3c5b4ce13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6261e7ddf494aaa966967ddb5a1fd44)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:34.699914 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ded053d-3088-408c-a01b-2df3c5b4ce13 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:34.792409 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5ded053d-3088-408c-a01b-2df3c5b4ce13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6261e7ddf494aaa966967ddb5a1fd44', 'request_id': 'req-5ded053d-3088-408c-a01b-2df3c5b4ce13'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:34.800950 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5ded053d-3088-408c-a01b-2df3c5b4ce13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6261e7ddf494aaa966967ddb5a1fd44', 'request_id': 'req-5ded053d-3088-408c-a01b-2df3c5b4ce13'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:34.811989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5ded053d-3088-408c-a01b-2df3c5b4ce13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6261e7ddf494aaa966967ddb5a1fd44', 'request_id': 'req-5ded053d-3088-408c-a01b-2df3c5b4ce13'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:34.836998 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5ded053d-3088-408c-a01b-2df3c5b4ce13 admin admin] The token cache is being invalidate because project e6261e7ddf494aaa966967ddb5a1fd44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:01:34.837290 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5ded053d-3088-408c-a01b-2df3c5b4ce13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:34.838061 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5ded053d-3088-408c-a01b-2df3c5b4ce13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:34.839689 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1247/2494] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:01:34 2022] DELETE /identity/v3/projects/e6261e7ddf494aaa966967ddb5a1fd44 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:01:34.844434 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d022aa1-ccc4-49a2-8c40-e2d7665532ad service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:34.855245 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d022aa1-ccc4-49a2-8c40-e2d7665532ad service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:34.903383 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:01:34.945535 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb69f70-563a-49ff-9b74-50f1608ddb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:34.946650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb69f70-563a-49ff-9b74-50f1608ddb7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:34.946944 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb69f70-563a-49ff-9b74-50f1608ddb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:34.947334 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb69f70-563a-49ff-9b74-50f1608ddb7f admin admin] PATH_INFO: `/v3/users/b2ec4b391ac7417394201a1a585ad775` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:34.949103 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb69f70-563a-49ff-9b74-50f1608ddb7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2ec4b391ac7417394201a1a585ad775)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:34.951890 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb69f70-563a-49ff-9b74-50f1608ddb7f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:35.013036 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6bb69f70-563a-49ff-9b74-50f1608ddb7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2ec4b391ac7417394201a1a585ad775'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:35.025390 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6bb69f70-563a-49ff-9b74-50f1608ddb7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2ec4b391ac7417394201a1a585ad775'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:35.034734 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6bb69f70-563a-49ff-9b74-50f1608ddb7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2ec4b391ac7417394201a1a585ad775'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:35.035483 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6bb69f70-563a-49ff-9b74-50f1608ddb7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2ec4b391ac7417394201a1a585ad775'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:35.048966 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6bb69f70-563a-49ff-9b74-50f1608ddb7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2ec4b391ac7417394201a1a585ad775'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:35.058845 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1248/2495] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:01:34 2022] DELETE /identity/v3/users/b2ec4b391ac7417394201a1a585ad775 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:01:35.400666 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ded053d-3088-408c-a01b-2df3c5b4ce13 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:35.401226 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5ded053d-3088-408c-a01b-2df3c5b4ce13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:35.423861 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ded053d-3088-408c-a01b-2df3c5b4ce13 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:35.503699 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2943504f-ba51-42f5-8b23-8763ea8052c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:35.504649 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943504f-ba51-42f5-8b23-8763ea8052c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:35.505107 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943504f-ba51-42f5-8b23-8763ea8052c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:35.505392 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943504f-ba51-42f5-8b23-8763ea8052c5 admin admin] PATH_INFO: `/v3/projects/3d0508b6d5e7459293e0897e3344a8e3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:35.506779 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2943504f-ba51-42f5-8b23-8763ea8052c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d0508b6d5e7459293e0897e3344a8e3)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:35.509351 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2943504f-ba51-42f5-8b23-8763ea8052c5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:35.563050 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2943504f-ba51-42f5-8b23-8763ea8052c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d0508b6d5e7459293e0897e3344a8e3', 'request_id': 'req-2943504f-ba51-42f5-8b23-8763ea8052c5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:35.569672 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2943504f-ba51-42f5-8b23-8763ea8052c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d0508b6d5e7459293e0897e3344a8e3', 'request_id': 'req-2943504f-ba51-42f5-8b23-8763ea8052c5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:35.575693 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2943504f-ba51-42f5-8b23-8763ea8052c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d0508b6d5e7459293e0897e3344a8e3', 'request_id': 'req-2943504f-ba51-42f5-8b23-8763ea8052c5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:35.593055 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2943504f-ba51-42f5-8b23-8763ea8052c5 admin admin] The token cache is being invalidate because project 3d0508b6d5e7459293e0897e3344a8e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:01:35.593455 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2943504f-ba51-42f5-8b23-8763ea8052c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:35.595233 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2943504f-ba51-42f5-8b23-8763ea8052c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:35.597086 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1248/2496] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:01:35 2022] DELETE /identity/v3/projects/3d0508b6d5e7459293e0897e3344a8e3 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:01:35.601299 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb69f70-563a-49ff-9b74-50f1608ddb7f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:35.603122 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6bb69f70-563a-49ff-9b74-50f1608ddb7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:35.618423 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb69f70-563a-49ff-9b74-50f1608ddb7f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:35.705460 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844c3a4a-b697-4f21-87df-e44067946a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:35.706300 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c3a4a-b697-4f21-87df-e44067946a58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:35.706584 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c3a4a-b697-4f21-87df-e44067946a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:35.707023 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c3a4a-b697-4f21-87df-e44067946a58 admin admin] PATH_INFO: `/v3/users/89970b8043c54cb6986482e917787808` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:35.708402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844c3a4a-b697-4f21-87df-e44067946a58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89970b8043c54cb6986482e917787808)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:35.710389 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844c3a4a-b697-4f21-87df-e44067946a58 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:35.770660 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-844c3a4a-b697-4f21-87df-e44067946a58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89970b8043c54cb6986482e917787808'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:35.775125 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:01:35.781747 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-844c3a4a-b697-4f21-87df-e44067946a58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89970b8043c54cb6986482e917787808'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:35.807480 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-844c3a4a-b697-4f21-87df-e44067946a58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89970b8043c54cb6986482e917787808'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:35.808566 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-844c3a4a-b697-4f21-87df-e44067946a58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89970b8043c54cb6986482e917787808'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:35.831723 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-844c3a4a-b697-4f21-87df-e44067946a58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89970b8043c54cb6986482e917787808'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:35.834316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1249/2497] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:01:35 2022] DELETE /identity/v3/users/89970b8043c54cb6986482e917787808 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 20:01:36.147932 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2943504f-ba51-42f5-8b23-8763ea8052c5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:36.149405 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2943504f-ba51-42f5-8b23-8763ea8052c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:36.164652 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2943504f-ba51-42f5-8b23-8763ea8052c5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:36.187739 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:01:36.246444 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39f87c-60d3-44ae-9342-ed3c204285de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:36.247387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39f87c-60d3-44ae-9342-ed3c204285de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:36.247700 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39f87c-60d3-44ae-9342-ed3c204285de admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:36.248070 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39f87c-60d3-44ae-9342-ed3c204285de admin admin] PATH_INFO: `/v3/projects/a071c8f2530d4bcea4efa92b6c31ea57` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:36.249435 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b39f87c-60d3-44ae-9342-ed3c204285de admin admin] RBAC: Authorizing `identity:delete_project(project_id=a071c8f2530d4bcea4efa92b6c31ea57)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:36.252020 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b39f87c-60d3-44ae-9342-ed3c204285de admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:36.313330 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7b39f87c-60d3-44ae-9342-ed3c204285de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a071c8f2530d4bcea4efa92b6c31ea57', 'request_id': 'req-7b39f87c-60d3-44ae-9342-ed3c204285de'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:36.324771 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7b39f87c-60d3-44ae-9342-ed3c204285de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a071c8f2530d4bcea4efa92b6c31ea57', 'request_id': 'req-7b39f87c-60d3-44ae-9342-ed3c204285de'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:36.333872 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7b39f87c-60d3-44ae-9342-ed3c204285de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a071c8f2530d4bcea4efa92b6c31ea57', 'request_id': 'req-7b39f87c-60d3-44ae-9342-ed3c204285de'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:36.355310 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7b39f87c-60d3-44ae-9342-ed3c204285de admin admin] The token cache is being invalidate because project a071c8f2530d4bcea4efa92b6c31ea57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:01:36.356317 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7b39f87c-60d3-44ae-9342-ed3c204285de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:36.358322 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7b39f87c-60d3-44ae-9342-ed3c204285de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:36.359948 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1249/2498] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:01:36 2022] DELETE /identity/v3/projects/a071c8f2530d4bcea4efa92b6c31ea57 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:01:36.365067 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b8695-1aee-4188-a775-934076a698e1 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:36.365605 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b8695-1aee-4188-a775-934076a698e1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:36.366225 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b8695-1aee-4188-a775-934076a698e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:36.389083 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ea4b8695-1aee-4188-a775-934076a698e1 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:01:36.391096 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ea4b8695-1aee-4188-a775-934076a698e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:36.473335 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1250/2499] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:01:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:01:36.481398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc42fb3-2767-4a87-8277-5c36791b3ba4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:36.481955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc42fb3-2767-4a87-8277-5c36791b3ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:36.482378 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc42fb3-2767-4a87-8277-5c36791b3ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:36.517105 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0dc42fb3-2767-4a87-8277-5c36791b3ba4 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:01:36.518991 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0dc42fb3-2767-4a87-8277-5c36791b3ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:36.542673 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1250/2500] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:01:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:01:36.547811 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b8695-1aee-4188-a775-934076a698e1 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:36.559469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b8695-1aee-4188-a775-934076a698e1 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:36.565911 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edfad69-5618-43a7-aa11-f502cccdbf72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:36.566799 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfad69-5618-43a7-aa11-f502cccdbf72 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:36.567088 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfad69-5618-43a7-aa11-f502cccdbf72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:36.567367 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfad69-5618-43a7-aa11-f502cccdbf72 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:36.568158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edfad69-5618-43a7-aa11-f502cccdbf72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 20:01:36.568470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edfad69-5618-43a7-aa11-f502cccdbf72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:36.570104 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edfad69-5618-43a7-aa11-f502cccdbf72 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:36.583407 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1251/2501] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 20:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 20:01:36.588351 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc42fb3-2767-4a87-8277-5c36791b3ba4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:36.598910 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc42fb3-2767-4a87-8277-5c36791b3ba4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:36.605321 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36218f99-38fa-464f-85f3-6ca893f43bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:36.606008 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36218f99-38fa-464f-85f3-6ca893f43bf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:36.606294 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36218f99-38fa-464f-85f3-6ca893f43bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:36.606574 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36218f99-38fa-464f-85f3-6ca893f43bf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:36.607284 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36218f99-38fa-464f-85f3-6ca893f43bf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:36.609376 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36218f99-38fa-464f-85f3-6ca893f43bf5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:36.645251 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1251/2502] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:01:36 2022] POST /identity/v3/projects => generated 387 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:01:36.650199 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edfad69-5618-43a7-aa11-f502cccdbf72 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:36.665516 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edfad69-5618-43a7-aa11-f502cccdbf72 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:36.728740 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e31079-a92c-40e9-a43e-3e9fde8542da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:36.729420 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e31079-a92c-40e9-a43e-3e9fde8542da admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:36.729704 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e31079-a92c-40e9-a43e-3e9fde8542da admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:36.729980 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e31079-a92c-40e9-a43e-3e9fde8542da admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:36.730671 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e31079-a92c-40e9-a43e-3e9fde8542da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:36.732664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e31079-a92c-40e9-a43e-3e9fde8542da admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:36.758409 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1252/2503] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:01:36 2022] POST /identity/v3/users => generated 294 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:01:36.768768 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36218f99-38fa-464f-85f3-6ca893f43bf5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:36.787548 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36218f99-38fa-464f-85f3-6ca893f43bf5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:36.792426 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b6d14-a224-4676-9eb7-919dc568d418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:36.793109 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b6d14-a224-4676-9eb7-919dc568d418 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:36.793426 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b6d14-a224-4676-9eb7-919dc568d418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:36.793705 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b6d14-a224-4676-9eb7-919dc568d418 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:36.794444 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08b6d14-a224-4676-9eb7-919dc568d418 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:36.796110 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08b6d14-a224-4676-9eb7-919dc568d418 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:36.806929 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1252/2504] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 20:01:36.811169 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e31079-a92c-40e9-a43e-3e9fde8542da admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:36.822707 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e31079-a92c-40e9-a43e-3e9fde8542da admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:36.829120 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef4a83-3716-47ad-8a2e-99b75918588b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:36.829812 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef4a83-3716-47ad-8a2e-99b75918588b admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:36.830099 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef4a83-3716-47ad-8a2e-99b75918588b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:36.830387 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef4a83-3716-47ad-8a2e-99b75918588b admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:36.831134 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef4a83-3716-47ad-8a2e-99b75918588b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:36.832834 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef4a83-3716-47ad-8a2e-99b75918588b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:36.845960 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1253/2505] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 20:01:36.850103 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b6d14-a224-4676-9eb7-919dc568d418 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:36.862037 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b6d14-a224-4676-9eb7-919dc568d418 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:36.869894 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8dcf31-dc98-4082-9e32-e4a145e7418f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:36.871050 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8dcf31-dc98-4082-9e32-e4a145e7418f admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:36.871391 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8dcf31-dc98-4082-9e32-e4a145e7418f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:36.871720 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8dcf31-dc98-4082-9e32-e4a145e7418f admin admin] PATH_INFO: `/v3/projects/2baec59ccb724ef688ceb39bcbbb0b20/users/550269b247244797a2f92b0922a49cd2/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:36.890224 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8dcf31-dc98-4082-9e32-e4a145e7418f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2baec59ccb724ef688ceb39bcbbb0b20, user_id=550269b247244797a2f92b0922a49cd2, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:36.893209 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8dcf31-dc98-4082-9e32-e4a145e7418f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:36.911340 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1253/2506] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:01:36 2022] PUT /identity/v3/projects/2baec59ccb724ef688ceb39bcbbb0b20/users/550269b247244797a2f92b0922a49cd2/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:01:36.925353 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f47a7-515e-48bc-96d1-a8da44a05ac7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:36.925921 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f47a7-515e-48bc-96d1-a8da44a05ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:36.926203 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f47a7-515e-48bc-96d1-a8da44a05ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:36.960605 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d04f47a7-515e-48bc-96d1-a8da44a05ac7 None None] MFA Rules not processed for user `550269b247244797a2f92b0922a49cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:01:36.961844 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d04f47a7-515e-48bc-96d1-a8da44a05ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:36.990917 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:01:37.046273 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1254/2507] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:01:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:01:37.061504 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8dcf31-dc98-4082-9e32-e4a145e7418f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:37.063294 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-de8dcf31-dc98-4082-9e32-e4a145e7418f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:37.117021 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:01:37.159355 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8dcf31-dc98-4082-9e32-e4a145e7418f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:37.163375 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bab91c-e91f-4dac-9c1f-c2ecfd5682c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:37.167573 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bab91c-e91f-4dac-9c1f-c2ecfd5682c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:37.167573 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bab91c-e91f-4dac-9c1f-c2ecfd5682c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:37.167573 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bab91c-e91f-4dac-9c1f-c2ecfd5682c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:37.181223 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bab91c-e91f-4dac-9c1f-c2ecfd5682c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:37.182854 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bab91c-e91f-4dac-9c1f-c2ecfd5682c4 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:37.194517 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1254/2508] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:01:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:01:38.338283 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f47a7-515e-48bc-96d1-a8da44a05ac7 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:38.339896 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d04f47a7-515e-48bc-96d1-a8da44a05ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:38.424182 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f47a7-515e-48bc-96d1-a8da44a05ac7 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:38.427909 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aadb5f5-d0b8-4c70-a80d-94d255c95a1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:38.428580 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aadb5f5-d0b8-4c70-a80d-94d255c95a1b service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:38.428868 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aadb5f5-d0b8-4c70-a80d-94d255c95a1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:38.429151 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aadb5f5-d0b8-4c70-a80d-94d255c95a1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:38.430988 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7aadb5f5-d0b8-4c70-a80d-94d255c95a1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:38.441662 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aadb5f5-d0b8-4c70-a80d-94d255c95a1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:38.443717 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aadb5f5-d0b8-4c70-a80d-94d255c95a1b service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:38.454359 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1255/2509] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:01:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:01:47.189673 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=11 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:01:48.448158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=46 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:01:56.870293 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bab91c-e91f-4dac-9c1f-c2ecfd5682c4 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:56.944269 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bab91c-e91f-4dac-9c1f-c2ecfd5682c4 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:56.948925 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c10c41-8731-4b12-9794-7cc047f98eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:56.949807 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c10c41-8731-4b12-9794-7cc047f98eb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:56.950091 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c10c41-8731-4b12-9794-7cc047f98eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:56.950367 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c10c41-8731-4b12-9794-7cc047f98eb8 admin admin] PATH_INFO: `/v3/users/550269b247244797a2f92b0922a49cd2` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:56.951836 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c10c41-8731-4b12-9794-7cc047f98eb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=550269b247244797a2f92b0922a49cd2)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:56.954559 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c10c41-8731-4b12-9794-7cc047f98eb8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:57.007837 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a6c10c41-8731-4b12-9794-7cc047f98eb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '550269b247244797a2f92b0922a49cd2'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:57.014803 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a6c10c41-8731-4b12-9794-7cc047f98eb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '550269b247244797a2f92b0922a49cd2'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:57.025655 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a6c10c41-8731-4b12-9794-7cc047f98eb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '550269b247244797a2f92b0922a49cd2'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:57.026319 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a6c10c41-8731-4b12-9794-7cc047f98eb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '550269b247244797a2f92b0922a49cd2'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:57.026973 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a6c10c41-8731-4b12-9794-7cc047f98eb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '550269b247244797a2f92b0922a49cd2'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:57.048398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1255/2510] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:01:56 2022] DELETE /identity/v3/users/550269b247244797a2f92b0922a49cd2 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:01:57.053331 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b36b3-64f3-485f-a23d-be944867747c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:57.053889 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b36b3-64f3-485f-a23d-be944867747c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:57.054017 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b36b3-64f3-485f-a23d-be944867747c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:57.074892 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8b0b36b3-64f3-485f-a23d-be944867747c None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:01:57.076158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8b0b36b3-64f3-485f-a23d-be944867747c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:57.143304 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1256/2511] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:01:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:01:57.153111 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c10c41-8731-4b12-9794-7cc047f98eb8 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:57.154555 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a6c10c41-8731-4b12-9794-7cc047f98eb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:57.220469 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c10c41-8731-4b12-9794-7cc047f98eb8 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:57.224254 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dcb5d3-5a41-41e3-8f83-98d44b3dc557 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:57.224917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dcb5d3-5a41-41e3-8f83-98d44b3dc557 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:57.225203 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dcb5d3-5a41-41e3-8f83-98d44b3dc557 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:57.225485 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dcb5d3-5a41-41e3-8f83-98d44b3dc557 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:57.238018 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dcb5d3-5a41-41e3-8f83-98d44b3dc557 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:57.239481 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dcb5d3-5a41-41e3-8f83-98d44b3dc557 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:57.250906 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1256/2512] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:01:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:01:57.544723 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b36b3-64f3-485f-a23d-be944867747c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:57.546255 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8b0b36b3-64f3-485f-a23d-be944867747c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:57.559777 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b36b3-64f3-485f-a23d-be944867747c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:57.567140 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c96dbc-a10c-4443-8183-f80811704ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:57.568958 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c96dbc-a10c-4443-8183-f80811704ee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:57.569510 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c96dbc-a10c-4443-8183-f80811704ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:57.570085 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c96dbc-a10c-4443-8183-f80811704ee7 admin admin] PATH_INFO: `/v3/projects/2baec59ccb724ef688ceb39bcbbb0b20` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:57.571951 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c96dbc-a10c-4443-8183-f80811704ee7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2baec59ccb724ef688ceb39bcbbb0b20)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:57.574854 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c96dbc-a10c-4443-8183-f80811704ee7 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:57.636285 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e0c96dbc-a10c-4443-8183-f80811704ee7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2baec59ccb724ef688ceb39bcbbb0b20', 'request_id': 'req-e0c96dbc-a10c-4443-8183-f80811704ee7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:57.646927 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e0c96dbc-a10c-4443-8183-f80811704ee7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2baec59ccb724ef688ceb39bcbbb0b20', 'request_id': 'req-e0c96dbc-a10c-4443-8183-f80811704ee7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:57.652981 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e0c96dbc-a10c-4443-8183-f80811704ee7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2baec59ccb724ef688ceb39bcbbb0b20', 'request_id': 'req-e0c96dbc-a10c-4443-8183-f80811704ee7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:57.664884 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e0c96dbc-a10c-4443-8183-f80811704ee7 admin admin] The token cache is being invalidate because project 2baec59ccb724ef688ceb39bcbbb0b20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:01:57.665172 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e0c96dbc-a10c-4443-8183-f80811704ee7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:57.665832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e0c96dbc-a10c-4443-8183-f80811704ee7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:01:57.667387 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1257/2513] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:01:57 2022] DELETE /identity/v3/projects/2baec59ccb724ef688ceb39bcbbb0b20 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:01:57.672610 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9947b9-dc04-427f-9514-3b21fe600bf3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:57.673012 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9947b9-dc04-427f-9514-3b21fe600bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:57.673181 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9947b9-dc04-427f-9514-3b21fe600bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:57.695010 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3e9947b9-dc04-427f-9514-3b21fe600bf3 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:01:57.696300 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3e9947b9-dc04-427f-9514-3b21fe600bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:57.719453 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:01:57.769463 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1257/2514] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:01:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:01:57.777865 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d917ebf-6e1c-4aba-9040-96ea93555fb2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:57.778153 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d917ebf-6e1c-4aba-9040-96ea93555fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:57.778439 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d917ebf-6e1c-4aba-9040-96ea93555fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:57.800195 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6d917ebf-6e1c-4aba-9040-96ea93555fb2 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:01:57.801444 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6d917ebf-6e1c-4aba-9040-96ea93555fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:57.810380 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1258/2515] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:01:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:01:57.815936 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9947b9-dc04-427f-9514-3b21fe600bf3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:57.825874 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9947b9-dc04-427f-9514-3b21fe600bf3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:57.830807 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f806975-b97e-41ac-9917-fed5a5afc8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:57.831475 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f806975-b97e-41ac-9917-fed5a5afc8e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:57.831785 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f806975-b97e-41ac-9917-fed5a5afc8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:57.832061 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f806975-b97e-41ac-9917-fed5a5afc8e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:57.832799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f806975-b97e-41ac-9917-fed5a5afc8e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 20:01:57.833108 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f806975-b97e-41ac-9917-fed5a5afc8e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:57.834726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f806975-b97e-41ac-9917-fed5a5afc8e8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:57.848011 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1258/2516] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 20:01:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 20:01:57.852547 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d917ebf-6e1c-4aba-9040-96ea93555fb2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:57.863851 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d917ebf-6e1c-4aba-9040-96ea93555fb2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:57.868922 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90fb9c8-9c75-4582-b1db-f055a9f03a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:57.869597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90fb9c8-9c75-4582-b1db-f055a9f03a81 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:57.869887 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90fb9c8-9c75-4582-b1db-f055a9f03a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:57.870165 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90fb9c8-9c75-4582-b1db-f055a9f03a81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:57.870865 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90fb9c8-9c75-4582-b1db-f055a9f03a81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:57.872838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90fb9c8-9c75-4582-b1db-f055a9f03a81 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:57.908365 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1259/2517] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:01:57 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:01:57.912695 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f806975-b97e-41ac-9917-fed5a5afc8e8 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:57.924687 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f806975-b97e-41ac-9917-fed5a5afc8e8 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:57.988585 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb186044-0dfc-4e1f-8938-937505962946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:57.989365 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb186044-0dfc-4e1f-8938-937505962946 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:57.989772 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb186044-0dfc-4e1f-8938-937505962946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:57.990021 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb186044-0dfc-4e1f-8938-937505962946 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:57.991027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb186044-0dfc-4e1f-8938-937505962946 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:57.993027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb186044-0dfc-4e1f-8938-937505962946 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:58.019180 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1259/2518] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:01:57 2022] POST /identity/v3/users => generated 292 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:01:58.023455 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90fb9c8-9c75-4582-b1db-f055a9f03a81 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:58.035126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90fb9c8-9c75-4582-b1db-f055a9f03a81 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:58.040063 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f02e01-6e23-4f53-b748-7b39952fca03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:58.040768 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f02e01-6e23-4f53-b748-7b39952fca03 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:58.041055 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f02e01-6e23-4f53-b748-7b39952fca03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:58.041333 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f02e01-6e23-4f53-b748-7b39952fca03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:58.042073 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f02e01-6e23-4f53-b748-7b39952fca03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:58.043721 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f02e01-6e23-4f53-b748-7b39952fca03 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:58.054545 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1260/2519] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:01:58.058700 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb186044-0dfc-4e1f-8938-937505962946 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:58.069605 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb186044-0dfc-4e1f-8938-937505962946 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:58.076207 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99ea7f-90ce-41ec-ac67-96a1107ea468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:58.076923 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99ea7f-90ce-41ec-ac67-96a1107ea468 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:58.077185 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99ea7f-90ce-41ec-ac67-96a1107ea468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:58.077472 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99ea7f-90ce-41ec-ac67-96a1107ea468 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:58.078777 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e99ea7f-90ce-41ec-ac67-96a1107ea468 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:58.080473 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e99ea7f-90ce-41ec-ac67-96a1107ea468 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:58.095820 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1260/2520] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:01:58.097241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f02e01-6e23-4f53-b748-7b39952fca03 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:58.107852 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f02e01-6e23-4f53-b748-7b39952fca03 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:58.113972 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6c900-519e-4a06-9f7c-f4401c58fc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:58.114871 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6c900-519e-4a06-9f7c-f4401c58fc62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:58.115156 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6c900-519e-4a06-9f7c-f4401c58fc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:58.115433 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6c900-519e-4a06-9f7c-f4401c58fc62 admin admin] PATH_INFO: `/v3/projects/e56f16131082413896ec23ec60378226/users/a0fadb91a8b54ece9ece2b5756be399e/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:58.135009 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a6c900-519e-4a06-9f7c-f4401c58fc62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e56f16131082413896ec23ec60378226, user_id=a0fadb91a8b54ece9ece2b5756be399e, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:58.147841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a6c900-519e-4a06-9f7c-f4401c58fc62 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:58.153978 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1261/2521] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:01:58 2022] PUT /identity/v3/projects/e56f16131082413896ec23ec60378226/users/a0fadb91a8b54ece9ece2b5756be399e/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:01:58.159369 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6314f-ed49-4e9c-adc9-5821397d02a8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:58.159634 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6314f-ed49-4e9c-adc9-5821397d02a8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:58.159914 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6314f-ed49-4e9c-adc9-5821397d02a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:58.180984 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b7e6314f-ed49-4e9c-adc9-5821397d02a8 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:01:58.182275 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b7e6314f-ed49-4e9c-adc9-5821397d02a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:58.247379 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1261/2522] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:01:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:01:58.258089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6c900-519e-4a06-9f7c-f4401c58fc62 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:01:58.259921 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-46a6c900-519e-4a06-9f7c-f4401c58fc62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:01:58.289503 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:01:58.329832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6c900-519e-4a06-9f7c-f4401c58fc62 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:01:58.333783 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57049e3-8044-4cea-a8f5-089f90525a6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:01:58.334448 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57049e3-8044-4cea-a8f5-089f90525a6c service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:01:58.334741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57049e3-8044-4cea-a8f5-089f90525a6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:01:58.335020 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57049e3-8044-4cea-a8f5-089f90525a6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:01:58.345622 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57049e3-8044-4cea-a8f5-089f90525a6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:01:58.347093 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57049e3-8044-4cea-a8f5-089f90525a6c service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:01:58.361433 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1262/2523] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:01:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:02:03.051354 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e1b9f-98b0-41a0-9fe7-ecad1f4144cb None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:02:03.051917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e1b9f-98b0-41a0-9fe7-ecad1f4144cb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:02:03.051917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e1b9f-98b0-41a0-9fe7-ecad1f4144cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:02:03.101477 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6c7e1b9f-98b0-41a0-9fe7-ecad1f4144cb None None] MFA Rules not processed for user `a0fadb91a8b54ece9ece2b5756be399e`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:02:03.102738 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6c7e1b9f-98b0-41a0-9fe7-ecad1f4144cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:02:03.129502 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:02:03.192071 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1262/2524] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:02:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:02:03.196790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57049e3-8044-4cea-a8f5-089f90525a6c service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:02:03.207925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57049e3-8044-4cea-a8f5-089f90525a6c service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:02:03.212854 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c29484-5ed9-4c6d-ae3f-2c92ac6df803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:02:03.213528 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c29484-5ed9-4c6d-ae3f-2c92ac6df803 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:02:03.213824 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c29484-5ed9-4c6d-ae3f-2c92ac6df803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:02:03.214104 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c29484-5ed9-4c6d-ae3f-2c92ac6df803 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:02:03.214822 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c29484-5ed9-4c6d-ae3f-2c92ac6df803 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:02:03.216799 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c29484-5ed9-4c6d-ae3f-2c92ac6df803 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:02:03.255496 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e1b9f-98b0-41a0-9fe7-ecad1f4144cb None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:02:03.255869 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1263/2525] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:02:03 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:02:03.268775 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e1b9f-98b0-41a0-9fe7-ecad1f4144cb None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:02:03.350998 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e65904-f9a0-4951-9cf3-61c4c17be421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:02:03.351742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e65904-f9a0-4951-9cf3-61c4c17be421 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:02:03.352038 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e65904-f9a0-4951-9cf3-61c4c17be421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:02:03.352327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e65904-f9a0-4951-9cf3-61c4c17be421 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:02:03.353169 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e65904-f9a0-4951-9cf3-61c4c17be421 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:02:03.355146 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e65904-f9a0-4951-9cf3-61c4c17be421 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:02:03.382006 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1263/2526] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:02:03 2022] POST /identity/v3/users => generated 299 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:02:03.386115 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c29484-5ed9-4c6d-ae3f-2c92ac6df803 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:02:03.398277 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c29484-5ed9-4c6d-ae3f-2c92ac6df803 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:02:03.403830 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a3daa-8d2b-46a5-b00d-b9104b304cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:02:03.404533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a3daa-8d2b-46a5-b00d-b9104b304cfe admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:02:03.404821 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a3daa-8d2b-46a5-b00d-b9104b304cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:02:03.405104 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a3daa-8d2b-46a5-b00d-b9104b304cfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:02:03.406439 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97a3daa-8d2b-46a5-b00d-b9104b304cfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:02:03.408134 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97a3daa-8d2b-46a5-b00d-b9104b304cfe admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:02:03.419848 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1264/2527] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 20:02:03.425796 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e65904-f9a0-4951-9cf3-61c4c17be421 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:02:03.438849 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e65904-f9a0-4951-9cf3-61c4c17be421 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:02:03.443690 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a26c1-931d-491e-af58-f30e8e31e5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:02:03.444602 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a26c1-931d-491e-af58-f30e8e31e5fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:02:03.444893 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a26c1-931d-491e-af58-f30e8e31e5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:02:03.445176 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a26c1-931d-491e-af58-f30e8e31e5fb admin admin] PATH_INFO: `/v3/projects/5849865a174b40b28021a581ea387847/users/922c141108634d7384b53a92728ae852/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:02:03.463651 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a26c1-931d-491e-af58-f30e8e31e5fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5849865a174b40b28021a581ea387847, user_id=922c141108634d7384b53a92728ae852, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:02:03.466365 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a26c1-931d-491e-af58-f30e8e31e5fb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:02:03.489528 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1264/2528] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:02:03 2022] PUT /identity/v3/projects/5849865a174b40b28021a581ea387847/users/922c141108634d7384b53a92728ae852/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:02:03.491072 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a3daa-8d2b-46a5-b00d-b9104b304cfe admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:02:03.504982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a3daa-8d2b-46a5-b00d-b9104b304cfe admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:02:03.597287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028b214d-d509-47ab-936f-c12e532d697a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:02:03.597942 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b214d-d509-47ab-936f-c12e532d697a admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:02:03.598274 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b214d-d509-47ab-936f-c12e532d697a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:02:03.598700 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b214d-d509-47ab-936f-c12e532d697a admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:02:03.599374 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028b214d-d509-47ab-936f-c12e532d697a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:02:03.601052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028b214d-d509-47ab-936f-c12e532d697a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:02:03.614155 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1265/2529] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:02:03.618633 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a26c1-931d-491e-af58-f30e8e31e5fb admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:02:03.629847 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a26c1-931d-491e-af58-f30e8e31e5fb admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:02:03.634663 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb14be6-927a-4ab6-8572-c9fe0ed3c1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:02:03.635590 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb14be6-927a-4ab6-8572-c9fe0ed3c1e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:02:03.635914 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb14be6-927a-4ab6-8572-c9fe0ed3c1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:02:03.636190 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb14be6-927a-4ab6-8572-c9fe0ed3c1e9 admin admin] PATH_INFO: `/v3/projects/5849865a174b40b28021a581ea387847/users/922c141108634d7384b53a92728ae852/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:02:03.638227 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb14be6-927a-4ab6-8572-c9fe0ed3c1e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5849865a174b40b28021a581ea387847, user_id=922c141108634d7384b53a92728ae852, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:02:03.641582 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb14be6-927a-4ab6-8572-c9fe0ed3c1e9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:02:03.652740 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1265/2530] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:02:03 2022] PUT /identity/v3/projects/5849865a174b40b28021a581ea387847/users/922c141108634d7384b53a92728ae852/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:02:08.761125 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c675af15-4f87-4789-8daf-5d9d4864b5a8 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:02:08.761995 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c675af15-4f87-4789-8daf-5d9d4864b5a8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:02:08.762676 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c675af15-4f87-4789-8daf-5d9d4864b5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:02:08.799313 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c675af15-4f87-4789-8daf-5d9d4864b5a8 None None] MFA Rules not processed for user `922c141108634d7384b53a92728ae852`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:02:08.800683 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c675af15-4f87-4789-8daf-5d9d4864b5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:02:08.900821 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1266/2531] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:02:08 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:02:08.914783 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb14be6-927a-4ab6-8572-c9fe0ed3c1e9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:02:08.916861 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2fb14be6-927a-4ab6-8572-c9fe0ed3c1e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:02:09.017150 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb14be6-927a-4ab6-8572-c9fe0ed3c1e9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:02:09.021893 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf44b5c-bf5d-4c49-9c78-3549cd3b8000 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98560723b49940d38a6a5e9cf3f5d3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:02:09.023653 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf44b5c-bf5d-4c49-9c78-3549cd3b8000 service glance] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:02:09.024204 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf44b5c-bf5d-4c49-9c78-3549cd3b8000 service glance] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:02:09.024437 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf44b5c-bf5d-4c49-9c78-3549cd3b8000 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:02:09.059118 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:02:09.103252 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf44b5c-bf5d-4c49-9c78-3549cd3b8000 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:02:09.104853 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf44b5c-bf5d-4c49-9c78-3549cd3b8000 service glance] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:02:09.124104 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1266/2532] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:02:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 20:02:09.268608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f0aad-5cca-4271-8129-f2cc42f21f11 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:02:09.272282 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f0aad-5cca-4271-8129-f2cc42f21f11 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:02:09.272924 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f0aad-5cca-4271-8129-f2cc42f21f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:02:09.301658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:02:09.361557 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-948f0aad-5cca-4271-8129-f2cc42f21f11 None None] MFA Rules not processed for user `1c0772205ae24713b8901666a87018b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:02:09.362729 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-948f0aad-5cca-4271-8129-f2cc42f21f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:02:09.453607 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1267/2533] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Jul 25 20:02:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:02:09.513360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf44b5c-bf5d-4c49-9c78-3549cd3b8000 service glance] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:02:09.514978 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5cf44b5c-bf5d-4c49-9c78-3549cd3b8000 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:02:09.637100 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf44b5c-bf5d-4c49-9c78-3549cd3b8000 service glance] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:02:09.642857 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ebc06-a12a-42be-92d2-4dcbdfc848cd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feafcf1b5a9844fe9738823fddd07e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:02:09.643536 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ebc06-a12a-42be-92d2-4dcbdfc848cd service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:02:09.643842 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ebc06-a12a-42be-92d2-4dcbdfc848cd service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:02:09.644121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ebc06-a12a-42be-92d2-4dcbdfc848cd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:02:09.653896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2ebc06-a12a-42be-92d2-4dcbdfc848cd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:02:09.656258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2ebc06-a12a-42be-92d2-4dcbdfc848cd service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:02:09.665731 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1267/2534] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Jul 25 20:02:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 20:02:18.701722 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36cbeb-efac-4b66-86a2-ed564b46f51a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:02:18.703000 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36cbeb-efac-4b66-86a2-ed564b46f51a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:02:18.703000 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36cbeb-efac-4b66-86a2-ed564b46f51a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:02:18.722923 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ab36cbeb-efac-4b66-86a2-ed564b46f51a None None] MFA Rules not processed for user `a0fadb91a8b54ece9ece2b5756be399e`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:02:18.724219 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ab36cbeb-efac-4b66-86a2-ed564b46f51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:02:18.752711 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1268/2535] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:02:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:02:19.026153 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ebc06-a12a-42be-92d2-4dcbdfc848cd service swift] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:02:19.027609 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4d2ebc06-a12a-42be-92d2-4dcbdfc848cd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:02:19.101009 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ebc06-a12a-42be-92d2-4dcbdfc848cd service swift] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:02:19.104762 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed53c2f-83f2-431d-a523-97f9952ecb70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:02:19.105319 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed53c2f-83f2-431d-a523-97f9952ecb70 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:02:19.105604 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed53c2f-83f2-431d-a523-97f9952ecb70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:02:19.105882 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed53c2f-83f2-431d-a523-97f9952ecb70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:02:19.107960 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7ed53c2f-83f2-431d-a523-97f9952ecb70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:02:19.119763 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed53c2f-83f2-431d-a523-97f9952ecb70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:02:19.121233 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed53c2f-83f2-431d-a523-97f9952ecb70 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:02:19.131960 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1268/2536] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:02:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:02:19.477954 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab36cbeb-efac-4b66-86a2-ed564b46f51a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:02:19.479483 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ab36cbeb-efac-4b66-86a2-ed564b46f51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:02:19.563809 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab36cbeb-efac-4b66-86a2-ed564b46f51a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:02:19.568817 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead7ba0-7e5c-45f9-a710-3861c08dd35f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0422d6695bf14925bf0f993cb16e64ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:02:19.569483 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead7ba0-7e5c-45f9-a710-3861c08dd35f service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:02:19.569765 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead7ba0-7e5c-45f9-a710-3861c08dd35f service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:02:19.570041 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead7ba0-7e5c-45f9-a710-3861c08dd35f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:02:19.571947 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7ead7ba0-7e5c-45f9-a710-3861c08dd35f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:02:19.586342 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead7ba0-7e5c-45f9-a710-3861c08dd35f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:02:19.588986 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead7ba0-7e5c-45f9-a710-3861c08dd35f service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:02:19.602667 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1269/2537] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:02:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:02:21.285164 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd97df5-bffd-4f68-ad1f-92c83086603f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:02:21.285728 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd97df5-bffd-4f68-ad1f-92c83086603f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:02:21.285728 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd97df5-bffd-4f68-ad1f-92c83086603f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:02:21.308344 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3bd97df5-bffd-4f68-ad1f-92c83086603f None None] MFA Rules not processed for user `1c0772205ae24713b8901666a87018b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:02:21.309606 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3bd97df5-bffd-4f68-ad1f-92c83086603f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:02:21.321153 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1269/2538] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Jul 25 20:02:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:02:21.335243 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead7ba0-7e5c-45f9-a710-3861c08dd35f service placement] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:02:21.348329 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead7ba0-7e5c-45f9-a710-3861c08dd35f service placement] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:02:21.353842 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f33b9-b425-4ff3-add9-06399ea46c30 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feafcf1b5a9844fe9738823fddd07e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:02:21.354513 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f33b9-b425-4ff3-add9-06399ea46c30 service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:02:21.354835 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f33b9-b425-4ff3-add9-06399ea46c30 service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:02:21.355125 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f33b9-b425-4ff3-add9-06399ea46c30 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:02:21.372099 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870f33b9-b425-4ff3-add9-06399ea46c30 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:02:21.373824 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870f33b9-b425-4ff3-add9-06399ea46c30 service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:02:21.386081 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1270/2539] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Jul 25 20:02:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 25 20:02:31.303042 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:02:31.380979 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=88 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:02:39.062954 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd97df5-bffd-4f68-ad1f-92c83086603f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:02:39.065630 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3bd97df5-bffd-4f68-ad1f-92c83086603f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:02:39.183041 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd97df5-bffd-4f68-ad1f-92c83086603f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:02:39.189858 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c425571-212c-435b-b0c7-40fb770aaf90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:02:39.190524 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c425571-212c-435b-b0c7-40fb770aaf90 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:02:39.190844 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c425571-212c-435b-b0c7-40fb770aaf90 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:02:39.191133 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c425571-212c-435b-b0c7-40fb770aaf90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:02:39.193029 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0c425571-212c-435b-b0c7-40fb770aaf90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:02:39.206517 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c425571-212c-435b-b0c7-40fb770aaf90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:02:39.208186 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c425571-212c-435b-b0c7-40fb770aaf90 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:02:39.220131 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1270/2540] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:02:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:02:49.214682 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=34 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:02:56.913097 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f33b9-b425-4ff3-add9-06399ea46c30 service swift] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:02:56.923502 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f33b9-b425-4ff3-add9-06399ea46c30 service swift] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:02:56.927498 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec80e5-4487-432e-b379-0feddf092ba2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:02:56.928181 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec80e5-4487-432e-b379-0feddf092ba2 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:02:56.928472 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec80e5-4487-432e-b379-0feddf092ba2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:02:56.928755 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec80e5-4487-432e-b379-0feddf092ba2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:02:56.939221 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ec80e5-4487-432e-b379-0feddf092ba2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:02:56.940584 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ec80e5-4487-432e-b379-0feddf092ba2 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:02:56.951859 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1271/2541] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:02:56 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:02:58.988535 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c425571-212c-435b-b0c7-40fb770aaf90 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:02:58.991926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0c425571-212c-435b-b0c7-40fb770aaf90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:02:59.025487 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c425571-212c-435b-b0c7-40fb770aaf90 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:02:59.038063 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a1bf0f-6125-4a9e-b2e0-18670add0aff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:02:59.039477 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1bf0f-6125-4a9e-b2e0-18670add0aff service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:02:59.040120 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1bf0f-6125-4a9e-b2e0-18670add0aff service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:02:59.040727 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1bf0f-6125-4a9e-b2e0-18670add0aff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:02:59.044082 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f9a1bf0f-6125-4a9e-b2e0-18670add0aff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:02:59.077261 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a1bf0f-6125-4a9e-b2e0-18670add0aff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:02:59.084208 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a1bf0f-6125-4a9e-b2e0-18670add0aff service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:02:59.112969 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1271/2542] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:02:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:03:06.947271 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:03:09.101577 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:03:11.418894 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec80e5-4487-432e-b379-0feddf092ba2 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:11.430033 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec80e5-4487-432e-b379-0feddf092ba2 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:11.434294 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68919c-bc95-4fc2-aff0-2f3910120ef1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:11.435107 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68919c-bc95-4fc2-aff0-2f3910120ef1 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:11.435491 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68919c-bc95-4fc2-aff0-2f3910120ef1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:11.435929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68919c-bc95-4fc2-aff0-2f3910120ef1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:11.438775 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0b68919c-bc95-4fc2-aff0-2f3910120ef1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:03:11.538778 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b68919c-bc95-4fc2-aff0-2f3910120ef1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:11.541460 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b68919c-bc95-4fc2-aff0-2f3910120ef1 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:11.558611 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1272/2543] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:03:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:03:14.902994 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a1bf0f-6125-4a9e-b2e0-18670add0aff service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:14.904869 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f9a1bf0f-6125-4a9e-b2e0-18670add0aff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:03:15.005006 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a1bf0f-6125-4a9e-b2e0-18670add0aff service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:15.009835 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb5e855-199a-47c0-aceb-8cb3d32a420d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:15.010807 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb5e855-199a-47c0-aceb-8cb3d32a420d service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:15.011270 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb5e855-199a-47c0-aceb-8cb3d32a420d service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:15.011966 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb5e855-199a-47c0-aceb-8cb3d32a420d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:15.016107 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-dcb5e855-199a-47c0-aceb-8cb3d32a420d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:03:15.145175 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb5e855-199a-47c0-aceb-8cb3d32a420d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:15.148236 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb5e855-199a-47c0-aceb-8cb3d32a420d service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:15.174931 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1272/2544] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:03:14 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:03:15.971802 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68919c-bc95-4fc2-aff0-2f3910120ef1 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:15.973813 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0b68919c-bc95-4fc2-aff0-2f3910120ef1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:03:15.990938 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68919c-bc95-4fc2-aff0-2f3910120ef1 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:15.996317 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa6a48d-6877-4606-94c0-99e4c3bd1ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:15.997277 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa6a48d-6877-4606-94c0-99e4c3bd1ffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:15.997601 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa6a48d-6877-4606-94c0-99e4c3bd1ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:15.998026 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa6a48d-6877-4606-94c0-99e4c3bd1ffe admin admin] PATH_INFO: `/v3/users/581ae6d40d34447da6651f3b2c21bdf0` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:15.999447 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa6a48d-6877-4606-94c0-99e4c3bd1ffe admin admin] RBAC: Authorizing `identity:delete_user(user_id=581ae6d40d34447da6651f3b2c21bdf0)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:16.001832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa6a48d-6877-4606-94c0-99e4c3bd1ffe admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:16.059845 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4fa6a48d-6877-4606-94c0-99e4c3bd1ffe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '581ae6d40d34447da6651f3b2c21bdf0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:16.070338 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4fa6a48d-6877-4606-94c0-99e4c3bd1ffe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '581ae6d40d34447da6651f3b2c21bdf0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:16.077857 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4fa6a48d-6877-4606-94c0-99e4c3bd1ffe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '581ae6d40d34447da6651f3b2c21bdf0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:16.079146 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4fa6a48d-6877-4606-94c0-99e4c3bd1ffe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '581ae6d40d34447da6651f3b2c21bdf0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:16.096667 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4fa6a48d-6877-4606-94c0-99e4c3bd1ffe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '581ae6d40d34447da6651f3b2c21bdf0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:16.103281 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1273/2545] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:03:15 2022] DELETE /identity/v3/users/581ae6d40d34447da6651f3b2c21bdf0 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:03:16.315637 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb5e855-199a-47c0-aceb-8cb3d32a420d service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:16.331829 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb5e855-199a-47c0-aceb-8cb3d32a420d service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:16.407234 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e397bb4-5842-4090-bd84-48551a6c3a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:16.408433 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e397bb4-5842-4090-bd84-48551a6c3a64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:16.408722 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e397bb4-5842-4090-bd84-48551a6c3a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:16.409052 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e397bb4-5842-4090-bd84-48551a6c3a64 admin admin] PATH_INFO: `/v3/projects/ac9a533f6f6f49e2bd73771fc99919c0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:16.415989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e397bb4-5842-4090-bd84-48551a6c3a64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac9a533f6f6f49e2bd73771fc99919c0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:16.418905 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e397bb4-5842-4090-bd84-48551a6c3a64 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:16.472965 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1e397bb4-5842-4090-bd84-48551a6c3a64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac9a533f6f6f49e2bd73771fc99919c0', 'request_id': 'req-1e397bb4-5842-4090-bd84-48551a6c3a64'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:16.479581 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1e397bb4-5842-4090-bd84-48551a6c3a64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac9a533f6f6f49e2bd73771fc99919c0', 'request_id': 'req-1e397bb4-5842-4090-bd84-48551a6c3a64'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:16.481728 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:03:16.487873 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1e397bb4-5842-4090-bd84-48551a6c3a64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac9a533f6f6f49e2bd73771fc99919c0', 'request_id': 'req-1e397bb4-5842-4090-bd84-48551a6c3a64'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:16.502460 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1e397bb4-5842-4090-bd84-48551a6c3a64 admin admin] The token cache is being invalidate because project ac9a533f6f6f49e2bd73771fc99919c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:03:16.502747 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1e397bb4-5842-4090-bd84-48551a6c3a64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:16.503402 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1e397bb4-5842-4090-bd84-48551a6c3a64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:16.505003 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1273/2546] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:03:16 2022] DELETE /identity/v3/projects/ac9a533f6f6f49e2bd73771fc99919c0 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:03:16.510469 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb06b01-a8dc-49a3-b6a7-b371775244c4 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:16.510771 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb06b01-a8dc-49a3-b6a7-b371775244c4 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:16.511058 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb06b01-a8dc-49a3-b6a7-b371775244c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:16.533420 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ceb06b01-a8dc-49a3-b6a7-b371775244c4 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:03:16.534732 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ceb06b01-a8dc-49a3-b6a7-b371775244c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:03:16.607647 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1274/2547] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:03:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:03:16.619702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b5cf0-364a-45e7-ae83-d1b1afcae3d9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:16.619929 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b5cf0-364a-45e7-ae83-d1b1afcae3d9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:16.620236 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b5cf0-364a-45e7-ae83-d1b1afcae3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:16.643878 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-649b5cf0-364a-45e7-ae83-d1b1afcae3d9 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:03:16.645279 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-649b5cf0-364a-45e7-ae83-d1b1afcae3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:03:16.654977 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1274/2548] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:03:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:03:16.664572 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb06b01-a8dc-49a3-b6a7-b371775244c4 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:16.675986 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb06b01-a8dc-49a3-b6a7-b371775244c4 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:16.680765 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6d411-1265-4e8f-a49a-21487d6f64bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:16.681442 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6d411-1265-4e8f-a49a-21487d6f64bd admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:16.681957 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6d411-1265-4e8f-a49a-21487d6f64bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:16.682315 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6d411-1265-4e8f-a49a-21487d6f64bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:16.683092 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b6d411-1265-4e8f-a49a-21487d6f64bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 20:03:16.683432 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b6d411-1265-4e8f-a49a-21487d6f64bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:16.685104 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b6d411-1265-4e8f-a49a-21487d6f64bd admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:16.689173 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:03:16.702703 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1275/2549] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 20:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 20:03:16.708880 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b5cf0-364a-45e7-ae83-d1b1afcae3d9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:16.721487 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b5cf0-364a-45e7-ae83-d1b1afcae3d9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:16.728900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6cc7c-a83c-4864-8a20-91be7bbed5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:16.730726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6cc7c-a83c-4864-8a20-91be7bbed5ae admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:16.731161 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6cc7c-a83c-4864-8a20-91be7bbed5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:16.731772 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6cc7c-a83c-4864-8a20-91be7bbed5ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:16.732873 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f6cc7c-a83c-4864-8a20-91be7bbed5ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:16.736240 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f6cc7c-a83c-4864-8a20-91be7bbed5ae admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:16.774328 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1275/2550] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:03:16 2022] POST /identity/v3/projects => generated 355 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:03:16.778885 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6d411-1265-4e8f-a49a-21487d6f64bd admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:16.790651 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6d411-1265-4e8f-a49a-21487d6f64bd admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:16.871490 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92774780-26b9-4aeb-b09e-55b3f79635b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:16.872592 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92774780-26b9-4aeb-b09e-55b3f79635b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:16.873102 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92774780-26b9-4aeb-b09e-55b3f79635b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:16.873616 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92774780-26b9-4aeb-b09e-55b3f79635b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:16.874985 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92774780-26b9-4aeb-b09e-55b3f79635b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:16.879538 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92774780-26b9-4aeb-b09e-55b3f79635b5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:16.909702 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1276/2551] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:03:16 2022] POST /identity/v3/users => generated 278 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:03:16.914481 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6cc7c-a83c-4864-8a20-91be7bbed5ae admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:16.928516 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6cc7c-a83c-4864-8a20-91be7bbed5ae admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:16.933639 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aa50ad-07ae-419b-8241-2dda8a5966f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:16.934332 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa50ad-07ae-419b-8241-2dda8a5966f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:16.934644 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa50ad-07ae-419b-8241-2dda8a5966f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:16.934906 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa50ad-07ae-419b-8241-2dda8a5966f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:16.935698 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aa50ad-07ae-419b-8241-2dda8a5966f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:16.937343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aa50ad-07ae-419b-8241-2dda8a5966f3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:16.949495 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1276/2552] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:03:16.954257 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92774780-26b9-4aeb-b09e-55b3f79635b5 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:16.971888 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92774780-26b9-4aeb-b09e-55b3f79635b5 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:16.979009 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714aa20-04ad-44ef-8a62-53d903a5f9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:16.980022 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714aa20-04ad-44ef-8a62-53d903a5f9f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:16.980395 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714aa20-04ad-44ef-8a62-53d903a5f9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:16.981013 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714aa20-04ad-44ef-8a62-53d903a5f9f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:16.981614 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a714aa20-04ad-44ef-8a62-53d903a5f9f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:16.986291 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a714aa20-04ad-44ef-8a62-53d903a5f9f0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:17.001051 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1277/2553] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:03:17.006003 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aa50ad-07ae-419b-8241-2dda8a5966f3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:17.023944 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aa50ad-07ae-419b-8241-2dda8a5966f3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:17.029484 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5e564-18b9-4a93-adcd-aeb1c21782b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:17.030392 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5e564-18b9-4a93-adcd-aeb1c21782b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:17.030688 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5e564-18b9-4a93-adcd-aeb1c21782b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:17.030970 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5e564-18b9-4a93-adcd-aeb1c21782b0 admin admin] PATH_INFO: `/v3/projects/1d752b523fa34b41a26310752e2f85a5/users/25aa50e0a29c440582ea28ffe0a19f1f/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:17.049341 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5e564-18b9-4a93-adcd-aeb1c21782b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d752b523fa34b41a26310752e2f85a5, user_id=25aa50e0a29c440582ea28ffe0a19f1f, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:17.052386 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5e564-18b9-4a93-adcd-aeb1c21782b0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:17.068043 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1277/2554] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:03:17 2022] PUT /identity/v3/projects/1d752b523fa34b41a26310752e2f85a5/users/25aa50e0a29c440582ea28ffe0a19f1f/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:03:17.139202 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbddf433-04c3-4352-9ac0-45e36c509b38 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:17.139637 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbddf433-04c3-4352-9ac0-45e36c509b38 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:17.139921 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbddf433-04c3-4352-9ac0-45e36c509b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:17.171545 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-fbddf433-04c3-4352-9ac0-45e36c509b38 None None] MFA Rules not processed for user `25aa50e0a29c440582ea28ffe0a19f1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:03:17.173364 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fbddf433-04c3-4352-9ac0-45e36c509b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:03:17.248001 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1278/2555] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:03:17 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:03:17.253162 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5e564-18b9-4a93-adcd-aeb1c21782b0 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:17.325580 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5e564-18b9-4a93-adcd-aeb1c21782b0 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:17.331182 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e834bc9-bbd3-42e5-88b2-edf1f1d1c20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:17.331902 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e834bc9-bbd3-42e5-88b2-edf1f1d1c20b admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:17.332197 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e834bc9-bbd3-42e5-88b2-edf1f1d1c20b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:17.332480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e834bc9-bbd3-42e5-88b2-edf1f1d1c20b admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:17.333198 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e834bc9-bbd3-42e5-88b2-edf1f1d1c20b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:17.335150 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e834bc9-bbd3-42e5-88b2-edf1f1d1c20b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:17.366833 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1278/2556] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:03:17 2022] POST /identity/v3/projects => generated 357 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:03:17.370997 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbddf433-04c3-4352-9ac0-45e36c509b38 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:17.383600 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbddf433-04c3-4352-9ac0-45e36c509b38 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:17.458283 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ce846-13ed-4855-80a0-2002fc7617ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:17.459644 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ce846-13ed-4855-80a0-2002fc7617ad admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:17.459995 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ce846-13ed-4855-80a0-2002fc7617ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:17.460520 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ce846-13ed-4855-80a0-2002fc7617ad admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:17.461805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ce846-13ed-4855-80a0-2002fc7617ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:17.465581 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ce846-13ed-4855-80a0-2002fc7617ad admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:17.482151 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:03:17.501524 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1279/2557] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:03:17 2022] POST /identity/v3/users => generated 285 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:03:17.505669 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e834bc9-bbd3-42e5-88b2-edf1f1d1c20b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:17.517605 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e834bc9-bbd3-42e5-88b2-edf1f1d1c20b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:17.522476 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4bb507-1c02-47a5-afb1-38a780862ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:17.523179 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4bb507-1c02-47a5-afb1-38a780862ede admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:17.523471 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4bb507-1c02-47a5-afb1-38a780862ede admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:17.523888 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4bb507-1c02-47a5-afb1-38a780862ede admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:17.524530 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4bb507-1c02-47a5-afb1-38a780862ede admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:17.526161 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4bb507-1c02-47a5-afb1-38a780862ede admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:17.538347 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1279/2558] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:03:17.543236 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ce846-13ed-4855-80a0-2002fc7617ad admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:17.555199 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ce846-13ed-4855-80a0-2002fc7617ad admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:17.560261 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3330c4f3-8e91-4efe-8638-fb1a50ee4851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:17.561166 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330c4f3-8e91-4efe-8638-fb1a50ee4851 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:17.561459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330c4f3-8e91-4efe-8638-fb1a50ee4851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:17.561743 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330c4f3-8e91-4efe-8638-fb1a50ee4851 admin admin] PATH_INFO: `/v3/projects/143a071f7b1040f98e4bf8b66f927d90/users/6868133ccd734c5b83acc58a5cf78f88/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:17.580908 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3330c4f3-8e91-4efe-8638-fb1a50ee4851 admin admin] RBAC: Authorizing `identity:create_grant(project_id=143a071f7b1040f98e4bf8b66f927d90, user_id=6868133ccd734c5b83acc58a5cf78f88, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:17.584860 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3330c4f3-8e91-4efe-8638-fb1a50ee4851 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:17.600665 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1280/2559] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:03:17 2022] PUT /identity/v3/projects/143a071f7b1040f98e4bf8b66f927d90/users/6868133ccd734c5b83acc58a5cf78f88/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:03:17.604842 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4bb507-1c02-47a5-afb1-38a780862ede admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:17.616792 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4bb507-1c02-47a5-afb1-38a780862ede admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:17.688515 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:03:17.699411 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c741128-9bfe-4881-ae60-ab5a14d1add1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:17.700434 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c741128-9bfe-4881-ae60-ab5a14d1add1 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:17.701271 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c741128-9bfe-4881-ae60-ab5a14d1add1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:17.701806 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c741128-9bfe-4881-ae60-ab5a14d1add1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:17.702443 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c741128-9bfe-4881-ae60-ab5a14d1add1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:17.705214 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c741128-9bfe-4881-ae60-ab5a14d1add1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:17.717528 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1280/2560] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:03:17.726057 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3330c4f3-8e91-4efe-8638-fb1a50ee4851 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:17.746740 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3330c4f3-8e91-4efe-8638-fb1a50ee4851 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:17.756719 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6414be0-71e2-42e4-98d6-d0a618dbbe95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:17.758345 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6414be0-71e2-42e4-98d6-d0a618dbbe95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:17.758828 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6414be0-71e2-42e4-98d6-d0a618dbbe95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:17.759337 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6414be0-71e2-42e4-98d6-d0a618dbbe95 admin admin] PATH_INFO: `/v3/projects/143a071f7b1040f98e4bf8b66f927d90/users/6868133ccd734c5b83acc58a5cf78f88/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:17.763125 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6414be0-71e2-42e4-98d6-d0a618dbbe95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=143a071f7b1040f98e4bf8b66f927d90, user_id=6868133ccd734c5b83acc58a5cf78f88, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:17.770815 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6414be0-71e2-42e4-98d6-d0a618dbbe95 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:17.789192 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1281/2561] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:03:17 2022] PUT /identity/v3/projects/143a071f7b1040f98e4bf8b66f927d90/users/6868133ccd734c5b83acc58a5cf78f88/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:03:17.803858 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8b06b-e4f9-4e88-96c7-d70d5ff61478 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:17.804307 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8b06b-e4f9-4e88-96c7-d70d5ff61478 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:17.804707 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8b06b-e4f9-4e88-96c7-d70d5ff61478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:17.843558 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-43c8b06b-e4f9-4e88-96c7-d70d5ff61478 None None] MFA Rules not processed for user `6868133ccd734c5b83acc58a5cf78f88`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:03:17.845529 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-43c8b06b-e4f9-4e88-96c7-d70d5ff61478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:03:17.961460 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1281/2562] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:03:17 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 20:03:17.978038 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6414be0-71e2-42e4-98d6-d0a618dbbe95 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:17.981100 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b6414be0-71e2-42e4-98d6-d0a618dbbe95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:03:18.058309 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6414be0-71e2-42e4-98d6-d0a618dbbe95 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:18.062660 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af676098-51b6-4e87-ba45-4a31b23e9426 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:18.064274 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af676098-51b6-4e87-ba45-4a31b23e9426 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:18.064615 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af676098-51b6-4e87-ba45-4a31b23e9426 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:18.064944 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af676098-51b6-4e87-ba45-4a31b23e9426 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:18.076533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af676098-51b6-4e87-ba45-4a31b23e9426 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:18.078100 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af676098-51b6-4e87-ba45-4a31b23e9426 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:18.090421 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1282/2563] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:03:17 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:03:18.204483 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c8b06b-e4f9-4e88-96c7-d70d5ff61478 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:18.216004 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c8b06b-e4f9-4e88-96c7-d70d5ff61478 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:18.220306 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef8332-74c1-464b-a243-8b1be3a57f5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:18.221018 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef8332-74c1-464b-a243-8b1be3a57f5f service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:18.221307 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef8332-74c1-464b-a243-8b1be3a57f5f service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:18.221585 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef8332-74c1-464b-a243-8b1be3a57f5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:18.307374 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef8332-74c1-464b-a243-8b1be3a57f5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:18.308996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef8332-74c1-464b-a243-8b1be3a57f5f service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:18.320672 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1282/2564] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:03:18 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:03:28.085393 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:INSERT=4,keystone:SELECT=43 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:03:28.321724 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=66 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:03:34.936626 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af676098-51b6-4e87-ba45-4a31b23e9426 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:35.029951 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af676098-51b6-4e87-ba45-4a31b23e9426 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:35.034945 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b06e25b-721e-4cd1-bcdc-adf02f97ebef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:35.035897 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b06e25b-721e-4cd1-bcdc-adf02f97ebef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:35.036182 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b06e25b-721e-4cd1-bcdc-adf02f97ebef admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:35.036459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b06e25b-721e-4cd1-bcdc-adf02f97ebef admin admin] PATH_INFO: `/v3/users/25aa50e0a29c440582ea28ffe0a19f1f` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:35.037752 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b06e25b-721e-4cd1-bcdc-adf02f97ebef admin admin] RBAC: Authorizing `identity:delete_user(user_id=25aa50e0a29c440582ea28ffe0a19f1f)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:35.039740 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b06e25b-721e-4cd1-bcdc-adf02f97ebef admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:35.101805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5b06e25b-721e-4cd1-bcdc-adf02f97ebef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25aa50e0a29c440582ea28ffe0a19f1f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:35.113654 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5b06e25b-721e-4cd1-bcdc-adf02f97ebef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25aa50e0a29c440582ea28ffe0a19f1f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:35.127132 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5b06e25b-721e-4cd1-bcdc-adf02f97ebef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25aa50e0a29c440582ea28ffe0a19f1f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:35.128487 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5b06e25b-721e-4cd1-bcdc-adf02f97ebef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25aa50e0a29c440582ea28ffe0a19f1f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:35.150999 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5b06e25b-721e-4cd1-bcdc-adf02f97ebef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25aa50e0a29c440582ea28ffe0a19f1f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:35.158966 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1283/2565] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:03:34 2022] DELETE /identity/v3/users/25aa50e0a29c440582ea28ffe0a19f1f => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 20:03:35.166395 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca33bc-71ce-4e95-8db1-dbedb24aa171 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:35.166395 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca33bc-71ce-4e95-8db1-dbedb24aa171 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:35.166395 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca33bc-71ce-4e95-8db1-dbedb24aa171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:35.208441 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a5ca33bc-71ce-4e95-8db1-dbedb24aa171 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:03:35.208902 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a5ca33bc-71ce-4e95-8db1-dbedb24aa171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:03:35.319547 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1283/2566] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:03:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:03:35.330467 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b06e25b-721e-4cd1-bcdc-adf02f97ebef admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:35.332094 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5b06e25b-721e-4cd1-bcdc-adf02f97ebef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:03:35.404864 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b06e25b-721e-4cd1-bcdc-adf02f97ebef admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:35.409570 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2b4ce-9858-4a57-a662-1c47421fa684 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:35.410271 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2b4ce-9858-4a57-a662-1c47421fa684 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:35.410519 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2b4ce-9858-4a57-a662-1c47421fa684 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:35.410801 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2b4ce-9858-4a57-a662-1c47421fa684 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:35.420866 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb2b4ce-9858-4a57-a662-1c47421fa684 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:35.423055 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb2b4ce-9858-4a57-a662-1c47421fa684 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:35.433961 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1284/2567] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:03:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:03:35.773077 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ca33bc-71ce-4e95-8db1-dbedb24aa171 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:35.774716 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a5ca33bc-71ce-4e95-8db1-dbedb24aa171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:03:35.792231 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ca33bc-71ce-4e95-8db1-dbedb24aa171 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:35.801926 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b0dbe-b9f0-4388-85f1-aa2ad65ed066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:35.803305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b0dbe-b9f0-4388-85f1-aa2ad65ed066 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:35.803955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b0dbe-b9f0-4388-85f1-aa2ad65ed066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:35.804492 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b0dbe-b9f0-4388-85f1-aa2ad65ed066 admin admin] PATH_INFO: `/v3/projects/1d752b523fa34b41a26310752e2f85a5` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:35.806905 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6b0dbe-b9f0-4388-85f1-aa2ad65ed066 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d752b523fa34b41a26310752e2f85a5)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:35.809991 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6b0dbe-b9f0-4388-85f1-aa2ad65ed066 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:35.887688 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2b6b0dbe-b9f0-4388-85f1-aa2ad65ed066 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d752b523fa34b41a26310752e2f85a5', 'request_id': 'req-2b6b0dbe-b9f0-4388-85f1-aa2ad65ed066'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:35.896109 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2b6b0dbe-b9f0-4388-85f1-aa2ad65ed066 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d752b523fa34b41a26310752e2f85a5', 'request_id': 'req-2b6b0dbe-b9f0-4388-85f1-aa2ad65ed066'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:35.903558 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2b6b0dbe-b9f0-4388-85f1-aa2ad65ed066 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d752b523fa34b41a26310752e2f85a5', 'request_id': 'req-2b6b0dbe-b9f0-4388-85f1-aa2ad65ed066'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:35.921749 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2b6b0dbe-b9f0-4388-85f1-aa2ad65ed066 admin admin] The token cache is being invalidate because project 1d752b523fa34b41a26310752e2f85a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:03:35.922919 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2b6b0dbe-b9f0-4388-85f1-aa2ad65ed066 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:35.923929 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2b6b0dbe-b9f0-4388-85f1-aa2ad65ed066 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:35.925914 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1284/2568] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:03:35 2022] DELETE /identity/v3/projects/1d752b523fa34b41a26310752e2f85a5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:03:35.931303 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2b4ce-9858-4a57-a662-1c47421fa684 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:35.946174 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2b4ce-9858-4a57-a662-1c47421fa684 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:35.972670 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:03:36.032578 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc06fb0e-b130-40af-9cc8-154403bc3032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:36.033675 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06fb0e-b130-40af-9cc8-154403bc3032 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:36.034015 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06fb0e-b130-40af-9cc8-154403bc3032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:36.034310 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06fb0e-b130-40af-9cc8-154403bc3032 admin admin] PATH_INFO: `/v3/users/6868133ccd734c5b83acc58a5cf78f88` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:36.035656 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc06fb0e-b130-40af-9cc8-154403bc3032 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6868133ccd734c5b83acc58a5cf78f88)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:36.037705 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc06fb0e-b130-40af-9cc8-154403bc3032 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:36.116929 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bc06fb0e-b130-40af-9cc8-154403bc3032 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6868133ccd734c5b83acc58a5cf78f88'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:36.127375 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bc06fb0e-b130-40af-9cc8-154403bc3032 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6868133ccd734c5b83acc58a5cf78f88'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:36.136768 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bc06fb0e-b130-40af-9cc8-154403bc3032 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6868133ccd734c5b83acc58a5cf78f88'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:36.137846 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bc06fb0e-b130-40af-9cc8-154403bc3032 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6868133ccd734c5b83acc58a5cf78f88'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:36.152766 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bc06fb0e-b130-40af-9cc8-154403bc3032 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6868133ccd734c5b83acc58a5cf78f88'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:36.164609 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1285/2569] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:03:35 2022] DELETE /identity/v3/users/6868133ccd734c5b83acc58a5cf78f88 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:03:36.655962 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b0dbe-b9f0-4388-85f1-aa2ad65ed066 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:36.655962 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2b6b0dbe-b9f0-4388-85f1-aa2ad65ed066 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:03:36.666428 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b0dbe-b9f0-4388-85f1-aa2ad65ed066 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:36.774050 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296912d2-1b6f-4f18-8610-413b90fdfe11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:36.774933 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296912d2-1b6f-4f18-8610-413b90fdfe11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:36.775217 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296912d2-1b6f-4f18-8610-413b90fdfe11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:36.775498 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296912d2-1b6f-4f18-8610-413b90fdfe11 admin admin] PATH_INFO: `/v3/projects/143a071f7b1040f98e4bf8b66f927d90` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:36.776842 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296912d2-1b6f-4f18-8610-413b90fdfe11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=143a071f7b1040f98e4bf8b66f927d90)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:36.779133 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296912d2-1b6f-4f18-8610-413b90fdfe11 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:36.816026 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:03:36.842617 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-296912d2-1b6f-4f18-8610-413b90fdfe11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '143a071f7b1040f98e4bf8b66f927d90', 'request_id': 'req-296912d2-1b6f-4f18-8610-413b90fdfe11'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:36.851708 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-296912d2-1b6f-4f18-8610-413b90fdfe11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '143a071f7b1040f98e4bf8b66f927d90', 'request_id': 'req-296912d2-1b6f-4f18-8610-413b90fdfe11'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:36.858064 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-296912d2-1b6f-4f18-8610-413b90fdfe11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '143a071f7b1040f98e4bf8b66f927d90', 'request_id': 'req-296912d2-1b6f-4f18-8610-413b90fdfe11'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:36.877746 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-296912d2-1b6f-4f18-8610-413b90fdfe11 admin admin] The token cache is being invalidate because project 143a071f7b1040f98e4bf8b66f927d90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:03:36.878225 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-296912d2-1b6f-4f18-8610-413b90fdfe11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:36.879284 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-296912d2-1b6f-4f18-8610-413b90fdfe11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:36.882031 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1285/2570] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:03:36 2022] DELETE /identity/v3/projects/143a071f7b1040f98e4bf8b66f927d90 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:03:36.887303 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b50d8c-ac5f-4c34-83f7-be380790f8fa None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:36.887902 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b50d8c-ac5f-4c34-83f7-be380790f8fa None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:36.888049 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b50d8c-ac5f-4c34-83f7-be380790f8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:36.919986 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c3b50d8c-ac5f-4c34-83f7-be380790f8fa None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:03:36.921443 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c3b50d8c-ac5f-4c34-83f7-be380790f8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:03:36.998267 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1286/2571] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:03:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 20:03:37.007341 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2350e7f-1523-444e-bab7-398aaa50c180 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:37.007682 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2350e7f-1523-444e-bab7-398aaa50c180 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:37.008003 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2350e7f-1523-444e-bab7-398aaa50c180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:37.030671 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f2350e7f-1523-444e-bab7-398aaa50c180 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:03:37.033087 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f2350e7f-1523-444e-bab7-398aaa50c180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:03:37.041585 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1286/2572] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:03:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:03:37.046085 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b50d8c-ac5f-4c34-83f7-be380790f8fa None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:37.063419 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b50d8c-ac5f-4c34-83f7-be380790f8fa None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:37.070145 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01802f-055b-4172-a458-9453bcba72ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:37.071224 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01802f-055b-4172-a458-9453bcba72ae admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:37.071640 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01802f-055b-4172-a458-9453bcba72ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:37.072113 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01802f-055b-4172-a458-9453bcba72ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:37.073529 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef01802f-055b-4172-a458-9453bcba72ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 20:03:37.074038 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef01802f-055b-4172-a458-9453bcba72ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:37.076950 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef01802f-055b-4172-a458-9453bcba72ae admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:37.096787 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1287/2573] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 20:03:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 20:03:37.104118 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2350e7f-1523-444e-bab7-398aaa50c180 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:37.121663 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2350e7f-1523-444e-bab7-398aaa50c180 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:37.129920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268170d1-ffb3-4653-b8ad-aad6f759352c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:37.130815 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268170d1-ffb3-4653-b8ad-aad6f759352c admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:37.131305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268170d1-ffb3-4653-b8ad-aad6f759352c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:37.131814 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268170d1-ffb3-4653-b8ad-aad6f759352c admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:37.132560 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268170d1-ffb3-4653-b8ad-aad6f759352c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:37.134746 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268170d1-ffb3-4653-b8ad-aad6f759352c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:37.170933 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1287/2574] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:03:37 2022] POST /identity/v3/projects => generated 369 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 20:03:37.179069 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01802f-055b-4172-a458-9453bcba72ae admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:37.195222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01802f-055b-4172-a458-9453bcba72ae admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:37.228058 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:03:37.269831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b724d-8b9a-4c1e-8ebc-63557c36533b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:37.270669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b724d-8b9a-4c1e-8ebc-63557c36533b admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:37.271104 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b724d-8b9a-4c1e-8ebc-63557c36533b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:37.271629 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b724d-8b9a-4c1e-8ebc-63557c36533b admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:37.272778 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b724d-8b9a-4c1e-8ebc-63557c36533b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:37.275620 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b724d-8b9a-4c1e-8ebc-63557c36533b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:37.306196 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1288/2575] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:03:37 2022] POST /identity/v3/users => generated 285 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 20:03:37.314359 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268170d1-ffb3-4653-b8ad-aad6f759352c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:37.331890 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268170d1-ffb3-4653-b8ad-aad6f759352c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:37.339601 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e8c4d-0c18-4cb6-bd01-d2e732345599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:37.340716 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e8c4d-0c18-4cb6-bd01-d2e732345599 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:37.340925 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e8c4d-0c18-4cb6-bd01-d2e732345599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:37.341194 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e8c4d-0c18-4cb6-bd01-d2e732345599 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:37.341940 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3e8c4d-0c18-4cb6-bd01-d2e732345599 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:37.344636 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3e8c4d-0c18-4cb6-bd01-d2e732345599 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:37.356755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1288/2576] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:03:37.362402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b724d-8b9a-4c1e-8ebc-63557c36533b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:37.373785 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b724d-8b9a-4c1e-8ebc-63557c36533b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:37.379857 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f140192-8153-4a54-8474-c8d0e26ea7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:37.380570 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f140192-8153-4a54-8474-c8d0e26ea7f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:37.380856 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f140192-8153-4a54-8474-c8d0e26ea7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:37.381174 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f140192-8153-4a54-8474-c8d0e26ea7f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:37.381955 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f140192-8153-4a54-8474-c8d0e26ea7f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:37.383703 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f140192-8153-4a54-8474-c8d0e26ea7f4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:37.401611 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1289/2577] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:03:37.406010 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e8c4d-0c18-4cb6-bd01-d2e732345599 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:37.417543 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e8c4d-0c18-4cb6-bd01-d2e732345599 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:37.423316 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b9d64-97e5-40c1-a90c-aafc077e47d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:37.424313 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b9d64-97e5-40c1-a90c-aafc077e47d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:37.424604 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b9d64-97e5-40c1-a90c-aafc077e47d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:37.424883 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b9d64-97e5-40c1-a90c-aafc077e47d8 admin admin] PATH_INFO: `/v3/projects/0b89f50595b74c93a9f1d358c07e786a/users/eead1d50759e48f3966600f498ad72d1/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:37.441541 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b9d64-97e5-40c1-a90c-aafc077e47d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b89f50595b74c93a9f1d358c07e786a, user_id=eead1d50759e48f3966600f498ad72d1, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:37.444348 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b9d64-97e5-40c1-a90c-aafc077e47d8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:37.460144 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1289/2578] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:03:37 2022] PUT /identity/v3/projects/0b89f50595b74c93a9f1d358c07e786a/users/eead1d50759e48f3966600f498ad72d1/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:03:37.472132 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35501c63-435a-4e5c-8f43-325f283adf62 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:37.472427 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35501c63-435a-4e5c-8f43-325f283adf62 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:37.472791 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35501c63-435a-4e5c-8f43-325f283adf62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:37.506144 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-35501c63-435a-4e5c-8f43-325f283adf62 None None] MFA Rules not processed for user `eead1d50759e48f3966600f498ad72d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:03:37.508307 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-35501c63-435a-4e5c-8f43-325f283adf62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:03:37.582910 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1290/2579] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:03:37 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:03:37.587800 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b9d64-97e5-40c1-a90c-aafc077e47d8 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:37.661543 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b9d64-97e5-40c1-a90c-aafc077e47d8 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:37.666658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2af9d-7d17-4e91-a2b7-2aeed8aea62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:37.667353 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2af9d-7d17-4e91-a2b7-2aeed8aea62d admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:37.667686 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2af9d-7d17-4e91-a2b7-2aeed8aea62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:37.667969 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2af9d-7d17-4e91-a2b7-2aeed8aea62d admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:37.668702 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f2af9d-7d17-4e91-a2b7-2aeed8aea62d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:37.670705 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f2af9d-7d17-4e91-a2b7-2aeed8aea62d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:37.707020 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1290/2580] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:03:37 2022] POST /identity/v3/projects => generated 367 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:03:37.713015 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35501c63-435a-4e5c-8f43-325f283adf62 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:37.728855 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35501c63-435a-4e5c-8f43-325f283adf62 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:37.829272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffb0e82-9735-41d0-b936-97e16b9f3034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:37.829609 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffb0e82-9735-41d0-b936-97e16b9f3034 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:37.829794 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffb0e82-9735-41d0-b936-97e16b9f3034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:37.830079 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffb0e82-9735-41d0-b936-97e16b9f3034 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:37.830942 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffb0e82-9735-41d0-b936-97e16b9f3034 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:37.833530 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffb0e82-9735-41d0-b936-97e16b9f3034 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:37.862336 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1291/2581] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:03:37 2022] POST /identity/v3/users => generated 290 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:03:37.867893 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2af9d-7d17-4e91-a2b7-2aeed8aea62d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:37.885304 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2af9d-7d17-4e91-a2b7-2aeed8aea62d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:37.892879 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae2466c-3eb1-4e36-8779-53e60f2c1303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:37.893782 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2466c-3eb1-4e36-8779-53e60f2c1303 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:37.894184 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2466c-3eb1-4e36-8779-53e60f2c1303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:37.894506 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2466c-3eb1-4e36-8779-53e60f2c1303 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:37.895431 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae2466c-3eb1-4e36-8779-53e60f2c1303 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:37.898462 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae2466c-3eb1-4e36-8779-53e60f2c1303 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:37.914587 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1291/2582] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:03:37.920000 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffb0e82-9735-41d0-b936-97e16b9f3034 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:37.933232 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffb0e82-9735-41d0-b936-97e16b9f3034 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:37.938858 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f68dd-b12c-43dd-9a27-eb62839c24e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:37.940003 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f68dd-b12c-43dd-9a27-eb62839c24e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:37.940342 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f68dd-b12c-43dd-9a27-eb62839c24e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:37.940588 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f68dd-b12c-43dd-9a27-eb62839c24e5 admin admin] PATH_INFO: `/v3/projects/5a8944d8eaf7430fa240b20713dc51e9/users/982be9b3886e4e7091120943b578afbb/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:37.961347 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f68dd-b12c-43dd-9a27-eb62839c24e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a8944d8eaf7430fa240b20713dc51e9, user_id=982be9b3886e4e7091120943b578afbb, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:37.964267 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f68dd-b12c-43dd-9a27-eb62839c24e5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:37.985119 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1292/2583] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:03:37 2022] PUT /identity/v3/projects/5a8944d8eaf7430fa240b20713dc51e9/users/982be9b3886e4e7091120943b578afbb/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:03:37.990765 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae2466c-3eb1-4e36-8779-53e60f2c1303 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:38.008823 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae2466c-3eb1-4e36-8779-53e60f2c1303 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:38.032948 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:03:38.087020 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfbdb73-e994-4f1b-9b0f-744271696261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:38.088053 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfbdb73-e994-4f1b-9b0f-744271696261 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:38.088543 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfbdb73-e994-4f1b-9b0f-744271696261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:38.088909 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfbdb73-e994-4f1b-9b0f-744271696261 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:38.090137 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfbdb73-e994-4f1b-9b0f-744271696261 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:38.092066 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfbdb73-e994-4f1b-9b0f-744271696261 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:38.108672 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1292/2584] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:03:38.113416 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f68dd-b12c-43dd-9a27-eb62839c24e5 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:38.122856 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:03:38.130960 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f68dd-b12c-43dd-9a27-eb62839c24e5 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:38.137334 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5e0782-e53a-40cc-a1e0-666ff2245f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:38.138291 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e0782-e53a-40cc-a1e0-666ff2245f3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:38.138577 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e0782-e53a-40cc-a1e0-666ff2245f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:38.138856 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e0782-e53a-40cc-a1e0-666ff2245f3c admin admin] PATH_INFO: `/v3/projects/5a8944d8eaf7430fa240b20713dc51e9/users/982be9b3886e4e7091120943b578afbb/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:38.141749 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5e0782-e53a-40cc-a1e0-666ff2245f3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a8944d8eaf7430fa240b20713dc51e9, user_id=982be9b3886e4e7091120943b578afbb, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:38.144668 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5e0782-e53a-40cc-a1e0-666ff2245f3c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:38.156963 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1293/2585] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:03:38 2022] PUT /identity/v3/projects/5a8944d8eaf7430fa240b20713dc51e9/users/982be9b3886e4e7091120943b578afbb/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:03:38.169071 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1097b6-dab7-46ca-90a0-1f2d91971998 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:38.169363 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1097b6-dab7-46ca-90a0-1f2d91971998 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:38.169650 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1097b6-dab7-46ca-90a0-1f2d91971998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:38.203116 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6b1097b6-dab7-46ca-90a0-1f2d91971998 None None] MFA Rules not processed for user `982be9b3886e4e7091120943b578afbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:03:38.205594 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6b1097b6-dab7-46ca-90a0-1f2d91971998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:03:38.288566 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1293/2586] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:03:38 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:03:38.309193 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5e0782-e53a-40cc-a1e0-666ff2245f3c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:38.311389 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cf5e0782-e53a-40cc-a1e0-666ff2245f3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:03:38.374013 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5e0782-e53a-40cc-a1e0-666ff2245f3c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:38.378357 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46dba62-b098-4115-a98a-8b54646148b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:38.379022 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46dba62-b098-4115-a98a-8b54646148b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:38.379322 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46dba62-b098-4115-a98a-8b54646148b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:38.379635 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46dba62-b098-4115-a98a-8b54646148b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:38.389947 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46dba62-b098-4115-a98a-8b54646148b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:38.392345 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46dba62-b098-4115-a98a-8b54646148b3 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:38.407498 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1294/2587] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:03:38 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:03:38.475802 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1097b6-dab7-46ca-90a0-1f2d91971998 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:38.482035 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1097b6-dab7-46ca-90a0-1f2d91971998 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:38.486063 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71afaf7-e84f-4257-b6bc-4b41878b3069 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:38.486737 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71afaf7-e84f-4257-b6bc-4b41878b3069 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:38.487022 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71afaf7-e84f-4257-b6bc-4b41878b3069 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:38.487292 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71afaf7-e84f-4257-b6bc-4b41878b3069 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:38.563294 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71afaf7-e84f-4257-b6bc-4b41878b3069 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:38.565732 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71afaf7-e84f-4257-b6bc-4b41878b3069 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:38.580120 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1294/2588] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:03:38 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:03:42.388259 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46dba62-b098-4115-a98a-8b54646148b3 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:42.473586 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46dba62-b098-4115-a98a-8b54646148b3 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:42.478586 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb484cc0-3b74-49db-ab10-1134cf344448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:42.479485 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb484cc0-3b74-49db-ab10-1134cf344448 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:42.479814 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb484cc0-3b74-49db-ab10-1134cf344448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:42.480096 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb484cc0-3b74-49db-ab10-1134cf344448 admin admin] PATH_INFO: `/v3/users/eead1d50759e48f3966600f498ad72d1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:42.481384 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb484cc0-3b74-49db-ab10-1134cf344448 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eead1d50759e48f3966600f498ad72d1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:42.483359 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb484cc0-3b74-49db-ab10-1134cf344448 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:42.546484 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fb484cc0-3b74-49db-ab10-1134cf344448 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eead1d50759e48f3966600f498ad72d1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:42.555915 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fb484cc0-3b74-49db-ab10-1134cf344448 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eead1d50759e48f3966600f498ad72d1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:42.570851 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fb484cc0-3b74-49db-ab10-1134cf344448 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eead1d50759e48f3966600f498ad72d1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:42.572077 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fb484cc0-3b74-49db-ab10-1134cf344448 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eead1d50759e48f3966600f498ad72d1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:42.597139 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fb484cc0-3b74-49db-ab10-1134cf344448 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eead1d50759e48f3966600f498ad72d1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:42.604339 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1295/2589] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:03:42 2022] DELETE /identity/v3/users/eead1d50759e48f3966600f498ad72d1 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:03:42.609350 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd75ca-4527-45f7-a3ef-116461a8891d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:42.610233 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd75ca-4527-45f7-a3ef-116461a8891d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:42.610233 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd75ca-4527-45f7-a3ef-116461a8891d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:42.631904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c7dd75ca-4527-45f7-a3ef-116461a8891d None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:03:42.633319 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c7dd75ca-4527-45f7-a3ef-116461a8891d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:03:42.693439 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:03:42.737003 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1295/2590] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:03:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 20:03:42.737579 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb484cc0-3b74-49db-ab10-1134cf344448 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:42.739386 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fb484cc0-3b74-49db-ab10-1134cf344448 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:03:42.767651 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:03:42.840196 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb484cc0-3b74-49db-ab10-1134cf344448 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:42.846721 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b7cea-3cc3-4c39-bfff-ae7b7d0fe085 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:42.847953 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b7cea-3cc3-4c39-bfff-ae7b7d0fe085 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:42.848459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b7cea-3cc3-4c39-bfff-ae7b7d0fe085 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:42.848936 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b7cea-3cc3-4c39-bfff-ae7b7d0fe085 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:42.865018 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6b7cea-3cc3-4c39-bfff-ae7b7d0fe085 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:42.866794 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6b7cea-3cc3-4c39-bfff-ae7b7d0fe085 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:42.880984 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1296/2591] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:03:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 20:03:43.320178 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd75ca-4527-45f7-a3ef-116461a8891d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:43.322050 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c7dd75ca-4527-45f7-a3ef-116461a8891d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:03:43.341909 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd75ca-4527-45f7-a3ef-116461a8891d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:43.352960 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba98e06-2c30-4eac-953b-d2c72a0f5b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:43.354752 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba98e06-2c30-4eac-953b-d2c72a0f5b44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:43.355331 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba98e06-2c30-4eac-953b-d2c72a0f5b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:43.355889 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba98e06-2c30-4eac-953b-d2c72a0f5b44 admin admin] PATH_INFO: `/v3/projects/0b89f50595b74c93a9f1d358c07e786a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:43.359811 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba98e06-2c30-4eac-953b-d2c72a0f5b44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b89f50595b74c93a9f1d358c07e786a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:43.364933 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba98e06-2c30-4eac-953b-d2c72a0f5b44 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:43.440509 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-dba98e06-2c30-4eac-953b-d2c72a0f5b44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b89f50595b74c93a9f1d358c07e786a', 'request_id': 'req-dba98e06-2c30-4eac-953b-d2c72a0f5b44'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:43.448116 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-dba98e06-2c30-4eac-953b-d2c72a0f5b44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b89f50595b74c93a9f1d358c07e786a', 'request_id': 'req-dba98e06-2c30-4eac-953b-d2c72a0f5b44'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:43.459167 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-dba98e06-2c30-4eac-953b-d2c72a0f5b44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b89f50595b74c93a9f1d358c07e786a', 'request_id': 'req-dba98e06-2c30-4eac-953b-d2c72a0f5b44'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:43.476887 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-dba98e06-2c30-4eac-953b-d2c72a0f5b44 admin admin] The token cache is being invalidate because project 0b89f50595b74c93a9f1d358c07e786a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:03:43.477267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-dba98e06-2c30-4eac-953b-d2c72a0f5b44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:43.477991 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-dba98e06-2c30-4eac-953b-d2c72a0f5b44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:43.479720 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1296/2592] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:03:43 2022] DELETE /identity/v3/projects/0b89f50595b74c93a9f1d358c07e786a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 20:03:43.484146 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b7cea-3cc3-4c39-bfff-ae7b7d0fe085 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:43.495594 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b7cea-3cc3-4c39-bfff-ae7b7d0fe085 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:43.574668 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8287729-f590-49fb-a187-b12a7b21a232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:43.575773 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8287729-f590-49fb-a187-b12a7b21a232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:43.576187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8287729-f590-49fb-a187-b12a7b21a232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:43.576588 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8287729-f590-49fb-a187-b12a7b21a232 admin admin] PATH_INFO: `/v3/users/982be9b3886e4e7091120943b578afbb` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:43.578337 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8287729-f590-49fb-a187-b12a7b21a232 admin admin] RBAC: Authorizing `identity:delete_user(user_id=982be9b3886e4e7091120943b578afbb)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:43.580768 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8287729-f590-49fb-a187-b12a7b21a232 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:43.657808 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d8287729-f590-49fb-a187-b12a7b21a232 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '982be9b3886e4e7091120943b578afbb'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:43.671264 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d8287729-f590-49fb-a187-b12a7b21a232 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '982be9b3886e4e7091120943b578afbb'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:43.681324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d8287729-f590-49fb-a187-b12a7b21a232 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '982be9b3886e4e7091120943b578afbb'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:43.682295 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d8287729-f590-49fb-a187-b12a7b21a232 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '982be9b3886e4e7091120943b578afbb'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:43.704746 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d8287729-f590-49fb-a187-b12a7b21a232 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '982be9b3886e4e7091120943b578afbb'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:43.715725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1297/2593] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:03:43 2022] DELETE /identity/v3/users/982be9b3886e4e7091120943b578afbb => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:03:44.200773 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba98e06-2c30-4eac-953b-d2c72a0f5b44 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:44.202390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-dba98e06-2c30-4eac-953b-d2c72a0f5b44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:03:44.218304 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba98e06-2c30-4eac-953b-d2c72a0f5b44 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:44.333587 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec014cae-6465-4115-b23e-f4628eff3c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:44.334647 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec014cae-6465-4115-b23e-f4628eff3c0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:44.334943 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec014cae-6465-4115-b23e-f4628eff3c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:44.335230 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec014cae-6465-4115-b23e-f4628eff3c0e admin admin] PATH_INFO: `/v3/projects/5a8944d8eaf7430fa240b20713dc51e9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:44.336626 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec014cae-6465-4115-b23e-f4628eff3c0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a8944d8eaf7430fa240b20713dc51e9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:44.338596 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec014cae-6465-4115-b23e-f4628eff3c0e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:44.392835 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ec014cae-6465-4115-b23e-f4628eff3c0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a8944d8eaf7430fa240b20713dc51e9', 'request_id': 'req-ec014cae-6465-4115-b23e-f4628eff3c0e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:44.400178 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ec014cae-6465-4115-b23e-f4628eff3c0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a8944d8eaf7430fa240b20713dc51e9', 'request_id': 'req-ec014cae-6465-4115-b23e-f4628eff3c0e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:44.406514 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ec014cae-6465-4115-b23e-f4628eff3c0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a8944d8eaf7430fa240b20713dc51e9', 'request_id': 'req-ec014cae-6465-4115-b23e-f4628eff3c0e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:44.408833 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:03:44.425588 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ec014cae-6465-4115-b23e-f4628eff3c0e admin admin] The token cache is being invalidate because project 5a8944d8eaf7430fa240b20713dc51e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:03:44.425913 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ec014cae-6465-4115-b23e-f4628eff3c0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:44.426709 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ec014cae-6465-4115-b23e-f4628eff3c0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:03:44.428234 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1297/2594] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:03:44 2022] DELETE /identity/v3/projects/5a8944d8eaf7430fa240b20713dc51e9 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:03:44.434320 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef14e84-d2ae-47d5-a0fd-9dbf4802e7d0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:44.435462 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef14e84-d2ae-47d5-a0fd-9dbf4802e7d0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:44.435462 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef14e84-d2ae-47d5-a0fd-9dbf4802e7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:44.467187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-aef14e84-d2ae-47d5-a0fd-9dbf4802e7d0 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:03:44.468664 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-aef14e84-d2ae-47d5-a0fd-9dbf4802e7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:03:44.507277 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:03:44.546618 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1298/2595] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:03:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:03:44.556138 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e174fc-166c-4c57-b5ea-26c2f1c5b7b3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:44.556527 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e174fc-166c-4c57-b5ea-26c2f1c5b7b3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:44.556817 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e174fc-166c-4c57-b5ea-26c2f1c5b7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:44.579987 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-06e174fc-166c-4c57-b5ea-26c2f1c5b7b3 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:03:44.581925 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-06e174fc-166c-4c57-b5ea-26c2f1c5b7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:03:44.591945 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1298/2596] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:03:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:03:44.602189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef14e84-d2ae-47d5-a0fd-9dbf4802e7d0 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:44.621708 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef14e84-d2ae-47d5-a0fd-9dbf4802e7d0 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:44.626486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5157c1-495d-44c0-b1d5-bf37b80e810d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:44.627233 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5157c1-495d-44c0-b1d5-bf37b80e810d admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:44.627539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5157c1-495d-44c0-b1d5-bf37b80e810d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:44.628183 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5157c1-495d-44c0-b1d5-bf37b80e810d admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:44.628892 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5157c1-495d-44c0-b1d5-bf37b80e810d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 20:03:44.629201 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5157c1-495d-44c0-b1d5-bf37b80e810d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:44.630958 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5157c1-495d-44c0-b1d5-bf37b80e810d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:44.644677 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1299/2597] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 20:03:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 20:03:44.649227 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e174fc-166c-4c57-b5ea-26c2f1c5b7b3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:44.660607 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e174fc-166c-4c57-b5ea-26c2f1c5b7b3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:44.665530 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f917cb6-7cc5-48d5-8881-07c6a016fa49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:44.666212 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f917cb6-7cc5-48d5-8881-07c6a016fa49 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:44.666498 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f917cb6-7cc5-48d5-8881-07c6a016fa49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:44.666785 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f917cb6-7cc5-48d5-8881-07c6a016fa49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:44.667489 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f917cb6-7cc5-48d5-8881-07c6a016fa49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:44.669486 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f917cb6-7cc5-48d5-8881-07c6a016fa49 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:44.700374 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1299/2598] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:03:44 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 20:03:44.704955 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5157c1-495d-44c0-b1d5-bf37b80e810d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:44.716800 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5157c1-495d-44c0-b1d5-bf37b80e810d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:44.778640 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6435064a-58ff-4824-89e7-80b1df8e70d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:44.779328 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6435064a-58ff-4824-89e7-80b1df8e70d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:44.779658 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6435064a-58ff-4824-89e7-80b1df8e70d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:44.779935 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6435064a-58ff-4824-89e7-80b1df8e70d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:44.780631 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6435064a-58ff-4824-89e7-80b1df8e70d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:44.782567 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6435064a-58ff-4824-89e7-80b1df8e70d7 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:44.810435 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1300/2599] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:03:44 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:03:44.812077 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f917cb6-7cc5-48d5-8881-07c6a016fa49 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:44.824904 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f917cb6-7cc5-48d5-8881-07c6a016fa49 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:44.829770 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4caa16a-231c-4849-90d1-4116d52f29fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:44.830997 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4caa16a-231c-4849-90d1-4116d52f29fb admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:44.831331 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4caa16a-231c-4849-90d1-4116d52f29fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:44.831655 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4caa16a-231c-4849-90d1-4116d52f29fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:44.832394 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4caa16a-231c-4849-90d1-4116d52f29fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:44.834023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4caa16a-231c-4849-90d1-4116d52f29fb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:44.848024 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1300/2600] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:03:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:03:44.852257 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6435064a-58ff-4824-89e7-80b1df8e70d7 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:44.863684 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6435064a-58ff-4824-89e7-80b1df8e70d7 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:44.869681 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ba6382-559b-49e8-8eef-642b866ed6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:44.870360 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba6382-559b-49e8-8eef-642b866ed6fc admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:44.870647 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba6382-559b-49e8-8eef-642b866ed6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:44.870935 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba6382-559b-49e8-8eef-642b866ed6fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:44.871709 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ba6382-559b-49e8-8eef-642b866ed6fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:44.873350 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ba6382-559b-49e8-8eef-642b866ed6fc admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:44.885715 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1301/2601] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:03:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:03:44.890619 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4caa16a-231c-4849-90d1-4116d52f29fb admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:44.902350 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4caa16a-231c-4849-90d1-4116d52f29fb admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:44.907390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819d7d50-6d9a-4f35-893b-14b3943cb131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:44.908409 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d7d50-6d9a-4f35-893b-14b3943cb131 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:44.908859 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d7d50-6d9a-4f35-893b-14b3943cb131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:44.909366 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d7d50-6d9a-4f35-893b-14b3943cb131 admin admin] PATH_INFO: `/v3/projects/14d08e4e98b541339426d511a8f1e8ba/users/b8017c2e5166464bac9425bd765dd544/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:44.929950 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819d7d50-6d9a-4f35-893b-14b3943cb131 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14d08e4e98b541339426d511a8f1e8ba, user_id=b8017c2e5166464bac9425bd765dd544, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:44.932719 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819d7d50-6d9a-4f35-893b-14b3943cb131 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:44.950243 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1301/2602] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:03:44 2022] PUT /identity/v3/projects/14d08e4e98b541339426d511a8f1e8ba/users/b8017c2e5166464bac9425bd765dd544/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:03:44.956301 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18bcdb-e193-42ff-96fd-a1184a258ae5 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:44.956688 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18bcdb-e193-42ff-96fd-a1184a258ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:44.957230 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18bcdb-e193-42ff-96fd-a1184a258ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:44.998362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-aa18bcdb-e193-42ff-96fd-a1184a258ae5 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:03:44.999733 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-aa18bcdb-e193-42ff-96fd-a1184a258ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:03:45.071820 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1302/2603] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:03:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:03:45.091261 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819d7d50-6d9a-4f35-893b-14b3943cb131 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:45.093065 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-819d7d50-6d9a-4f35-893b-14b3943cb131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:03:45.176625 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819d7d50-6d9a-4f35-893b-14b3943cb131 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:45.181740 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9698bf42-3084-4f50-8e35-9988a881ac33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:45.182406 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698bf42-3084-4f50-8e35-9988a881ac33 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:45.182695 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698bf42-3084-4f50-8e35-9988a881ac33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:45.182971 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698bf42-3084-4f50-8e35-9988a881ac33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:45.199982 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9698bf42-3084-4f50-8e35-9988a881ac33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:45.201840 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9698bf42-3084-4f50-8e35-9988a881ac33 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:45.218525 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1302/2604] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:03:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:03:46.218890 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7118d-cd79-4859-9262-ea00559a6a89 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:46.219439 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7118d-cd79-4859-9262-ea00559a6a89 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:46.219591 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7118d-cd79-4859-9262-ea00559a6a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:46.250126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-12b7118d-cd79-4859-9262-ea00559a6a89 None None] MFA Rules not processed for user `b8017c2e5166464bac9425bd765dd544`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:03:46.251862 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-12b7118d-cd79-4859-9262-ea00559a6a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:03:46.288265 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:03:46.332112 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1303/2605] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:03:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:03:46.336543 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9698bf42-3084-4f50-8e35-9988a881ac33 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:46.347270 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9698bf42-3084-4f50-8e35-9988a881ac33 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:46.352477 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47186677-fa8b-43ae-9a41-c7fdec0d5120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:46.353145 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47186677-fa8b-43ae-9a41-c7fdec0d5120 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:46.353431 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47186677-fa8b-43ae-9a41-c7fdec0d5120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:46.353708 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47186677-fa8b-43ae-9a41-c7fdec0d5120 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:46.354403 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47186677-fa8b-43ae-9a41-c7fdec0d5120 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:46.356450 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47186677-fa8b-43ae-9a41-c7fdec0d5120 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:46.391860 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1303/2606] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:03:46 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 20:03:46.396168 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7118d-cd79-4859-9262-ea00559a6a89 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:46.410080 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7118d-cd79-4859-9262-ea00559a6a89 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:46.479162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d84059-8a9c-4791-b211-358ff487fd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:46.480031 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d84059-8a9c-4791-b211-358ff487fd10 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:46.480404 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d84059-8a9c-4791-b211-358ff487fd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:46.480608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d84059-8a9c-4791-b211-358ff487fd10 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:46.481320 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d84059-8a9c-4791-b211-358ff487fd10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:46.483340 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d84059-8a9c-4791-b211-358ff487fd10 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:46.512908 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1304/2607] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:03:46 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:03:46.514431 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47186677-fa8b-43ae-9a41-c7fdec0d5120 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:46.533029 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47186677-fa8b-43ae-9a41-c7fdec0d5120 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:46.538655 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8632cc7-f74a-4ddc-99bc-1b4493fe66d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:46.539240 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8632cc7-f74a-4ddc-99bc-1b4493fe66d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:46.539534 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8632cc7-f74a-4ddc-99bc-1b4493fe66d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:46.539826 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8632cc7-f74a-4ddc-99bc-1b4493fe66d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:46.540561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8632cc7-f74a-4ddc-99bc-1b4493fe66d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:46.543033 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8632cc7-f74a-4ddc-99bc-1b4493fe66d2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:46.556042 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1304/2608] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:03:46.561030 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d84059-8a9c-4791-b211-358ff487fd10 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:46.586492 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d84059-8a9c-4791-b211-358ff487fd10 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:46.597214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aefaa70-1e24-477e-b2c8-eadf05088740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:46.598848 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aefaa70-1e24-477e-b2c8-eadf05088740 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:46.599462 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aefaa70-1e24-477e-b2c8-eadf05088740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:46.599996 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aefaa70-1e24-477e-b2c8-eadf05088740 admin admin] PATH_INFO: `/v3/projects/d0eb535f5d514a8cb0d79b8e757fec4c/users/6555c1eb8879440e8096459456dd4ebb/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:46.628283 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aefaa70-1e24-477e-b2c8-eadf05088740 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0eb535f5d514a8cb0d79b8e757fec4c, user_id=6555c1eb8879440e8096459456dd4ebb, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:46.631378 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aefaa70-1e24-477e-b2c8-eadf05088740 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:46.653723 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1305/2609] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:03:46 2022] PUT /identity/v3/projects/d0eb535f5d514a8cb0d79b8e757fec4c/users/6555c1eb8879440e8096459456dd4ebb/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:03:46.661271 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8632cc7-f74a-4ddc-99bc-1b4493fe66d2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:46.676270 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8632cc7-f74a-4ddc-99bc-1b4493fe66d2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:46.739947 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:03:46.751429 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a980a077-3125-49e1-b320-b644161be796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:46.752682 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a980a077-3125-49e1-b320-b644161be796 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:46.753331 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a980a077-3125-49e1-b320-b644161be796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:46.753684 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a980a077-3125-49e1-b320-b644161be796 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:46.754610 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a980a077-3125-49e1-b320-b644161be796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:46.756425 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a980a077-3125-49e1-b320-b644161be796 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:46.779060 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1305/2610] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:03:46.781428 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aefaa70-1e24-477e-b2c8-eadf05088740 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:46.794068 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aefaa70-1e24-477e-b2c8-eadf05088740 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:46.801583 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c701b9-c6b4-483f-8145-c8b770ff4000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:46.802655 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c701b9-c6b4-483f-8145-c8b770ff4000 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:46.803092 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c701b9-c6b4-483f-8145-c8b770ff4000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:46.803459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c701b9-c6b4-483f-8145-c8b770ff4000 admin admin] PATH_INFO: `/v3/projects/d0eb535f5d514a8cb0d79b8e757fec4c/users/6555c1eb8879440e8096459456dd4ebb/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:46.805692 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c701b9-c6b4-483f-8145-c8b770ff4000 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0eb535f5d514a8cb0d79b8e757fec4c, user_id=6555c1eb8879440e8096459456dd4ebb, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:46.809067 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c701b9-c6b4-483f-8145-c8b770ff4000 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:46.827396 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1306/2611] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:03:46 2022] PUT /identity/v3/projects/d0eb535f5d514a8cb0d79b8e757fec4c/users/6555c1eb8879440e8096459456dd4ebb/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:03:47.577547 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a980a077-3125-49e1-b320-b644161be796 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:47.579952 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a980a077-3125-49e1-b320-b644161be796 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:03:47.580184 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bd1e1-1bf2-4aef-a416-d66074f21e08 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:47.580591 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bd1e1-1bf2-4aef-a416-d66074f21e08 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:47.580681 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bd1e1-1bf2-4aef-a416-d66074f21e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:47.615647 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ad9bd1e1-1bf2-4aef-a416-d66074f21e08 None None] MFA Rules not processed for user `6555c1eb8879440e8096459456dd4ebb`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:03:47.617099 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ad9bd1e1-1bf2-4aef-a416-d66074f21e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:03:47.683287 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a980a077-3125-49e1-b320-b644161be796 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:47.729524 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1307/2612] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:03:47 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 20:03:47.745325 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9bd1e1-1bf2-4aef-a416-d66074f21e08 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:47.760459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ad9bd1e1-1bf2-4aef-a416-d66074f21e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:03:47.775005 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:03:47.822800 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f074d728-f935-4b5e-8f12-af72c248324e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0422d6695bf14925bf0f993cb16e64ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:47.823601 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074d728-f935-4b5e-8f12-af72c248324e service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:47.823903 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074d728-f935-4b5e-8f12-af72c248324e service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:47.824197 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074d728-f935-4b5e-8f12-af72c248324e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:47.826761 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f074d728-f935-4b5e-8f12-af72c248324e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:03:47.840693 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f074d728-f935-4b5e-8f12-af72c248324e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:47.842473 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f074d728-f935-4b5e-8f12-af72c248324e service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:47.855605 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1306/2613] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:03:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:03:47.870525 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9bd1e1-1bf2-4aef-a416-d66074f21e08 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:47.876427 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e500fa-c54f-48e2-a1c2-5f0e875d0760 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:47.878389 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e500fa-c54f-48e2-a1c2-5f0e875d0760 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:47.879016 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e500fa-c54f-48e2-a1c2-5f0e875d0760 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:47.879681 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e500fa-c54f-48e2-a1c2-5f0e875d0760 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:47.965537 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e500fa-c54f-48e2-a1c2-5f0e875d0760 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:47.967256 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e500fa-c54f-48e2-a1c2-5f0e875d0760 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:47.979248 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1308/2614] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:03:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 20:03:49.474789 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f074d728-f935-4b5e-8f12-af72c248324e service placement] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:03:49.476307 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f074d728-f935-4b5e-8f12-af72c248324e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:03:49.488355 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f074d728-f935-4b5e-8f12-af72c248324e service placement] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:03:49.492475 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2fa53b-ce1e-4903-ba6c-668f5d3d1db5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:03:49.493117 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2fa53b-ce1e-4903-ba6c-668f5d3d1db5 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:03:49.493401 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2fa53b-ce1e-4903-ba6c-668f5d3d1db5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:03:49.493678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2fa53b-ce1e-4903-ba6c-668f5d3d1db5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:03:49.505144 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2fa53b-ce1e-4903-ba6c-668f5d3d1db5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:03:49.506577 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2fa53b-ce1e-4903-ba6c-668f5d3d1db5 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:03:49.516864 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1307/2615] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:03:49 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:03:57.974493 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=45 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:03:59.512423 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=71 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:04:02.426442 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e500fa-c54f-48e2-a1c2-5f0e875d0760 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:02.428217 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-01e500fa-c54f-48e2-a1c2-5f0e875d0760 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:04:02.443851 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e500fa-c54f-48e2-a1c2-5f0e875d0760 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:02.448833 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201abf6b-da35-412c-94cd-0988ab1eb4d3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0422d6695bf14925bf0f993cb16e64ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:02.449483 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201abf6b-da35-412c-94cd-0988ab1eb4d3 service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:02.449768 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201abf6b-da35-412c-94cd-0988ab1eb4d3 service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:02.450049 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201abf6b-da35-412c-94cd-0988ab1eb4d3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:02.451870 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-201abf6b-da35-412c-94cd-0988ab1eb4d3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:04:02.464888 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201abf6b-da35-412c-94cd-0988ab1eb4d3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:02.466698 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201abf6b-da35-412c-94cd-0988ab1eb4d3 service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:02.479068 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1309/2616] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:04:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:04:03.055076 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2fa53b-ce1e-4903-ba6c-668f5d3d1db5 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:03.135786 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2fa53b-ce1e-4903-ba6c-668f5d3d1db5 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:03.140463 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375f49c-8bf6-4dba-ab34-751a1a313580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:03.141384 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375f49c-8bf6-4dba-ab34-751a1a313580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:03.141670 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375f49c-8bf6-4dba-ab34-751a1a313580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:03.141953 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375f49c-8bf6-4dba-ab34-751a1a313580 admin admin] PATH_INFO: `/v3/users/b8017c2e5166464bac9425bd765dd544` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:03.144062 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6375f49c-8bf6-4dba-ab34-751a1a313580 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8017c2e5166464bac9425bd765dd544)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:03.146056 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6375f49c-8bf6-4dba-ab34-751a1a313580 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:03.207649 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6375f49c-8bf6-4dba-ab34-751a1a313580 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8017c2e5166464bac9425bd765dd544'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:03.218083 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6375f49c-8bf6-4dba-ab34-751a1a313580 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8017c2e5166464bac9425bd765dd544'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:03.229190 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6375f49c-8bf6-4dba-ab34-751a1a313580 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8017c2e5166464bac9425bd765dd544'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:03.230000 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6375f49c-8bf6-4dba-ab34-751a1a313580 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8017c2e5166464bac9425bd765dd544'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:03.230625 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6375f49c-8bf6-4dba-ab34-751a1a313580 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8017c2e5166464bac9425bd765dd544'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:03.255230 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1308/2617] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:04:03 2022] DELETE /identity/v3/users/b8017c2e5166464bac9425bd765dd544 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:04:03.482386 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201abf6b-da35-412c-94cd-0988ab1eb4d3 service placement] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:03.500139 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201abf6b-da35-412c-94cd-0988ab1eb4d3 service placement] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:03.579436 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da814d-7fe7-4c6a-a847-073c2e4f2a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:03.581988 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da814d-7fe7-4c6a-a847-073c2e4f2a56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:03.582394 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da814d-7fe7-4c6a-a847-073c2e4f2a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:03.583012 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da814d-7fe7-4c6a-a847-073c2e4f2a56 admin admin] PATH_INFO: `/v3/projects/14d08e4e98b541339426d511a8f1e8ba` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:03.584597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da814d-7fe7-4c6a-a847-073c2e4f2a56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14d08e4e98b541339426d511a8f1e8ba)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:03.587476 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da814d-7fe7-4c6a-a847-073c2e4f2a56 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:03.654320 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d2da814d-7fe7-4c6a-a847-073c2e4f2a56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14d08e4e98b541339426d511a8f1e8ba', 'request_id': 'req-d2da814d-7fe7-4c6a-a847-073c2e4f2a56'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:03.665240 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d2da814d-7fe7-4c6a-a847-073c2e4f2a56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14d08e4e98b541339426d511a8f1e8ba', 'request_id': 'req-d2da814d-7fe7-4c6a-a847-073c2e4f2a56'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:03.674857 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d2da814d-7fe7-4c6a-a847-073c2e4f2a56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14d08e4e98b541339426d511a8f1e8ba', 'request_id': 'req-d2da814d-7fe7-4c6a-a847-073c2e4f2a56'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:03.691928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d2da814d-7fe7-4c6a-a847-073c2e4f2a56 admin admin] The token cache is being invalidate because project 14d08e4e98b541339426d511a8f1e8ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:04:03.692506 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d2da814d-7fe7-4c6a-a847-073c2e4f2a56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:03.693680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d2da814d-7fe7-4c6a-a847-073c2e4f2a56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:03.696616 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1310/2618] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:04:03 2022] DELETE /identity/v3/projects/14d08e4e98b541339426d511a8f1e8ba => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 20:04:03.701406 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375f49c-8bf6-4dba-ab34-751a1a313580 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:03.703175 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6375f49c-8bf6-4dba-ab34-751a1a313580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:04:03.723408 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375f49c-8bf6-4dba-ab34-751a1a313580 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:03.812055 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d0aac-0693-4902-acc7-8dcacb481126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:03.813434 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d0aac-0693-4902-acc7-8dcacb481126 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:03.813824 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d0aac-0693-4902-acc7-8dcacb481126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:03.815153 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d0aac-0693-4902-acc7-8dcacb481126 admin admin] PATH_INFO: `/v3/users/6555c1eb8879440e8096459456dd4ebb` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:03.817087 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9d0aac-0693-4902-acc7-8dcacb481126 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6555c1eb8879440e8096459456dd4ebb)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:03.819866 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9d0aac-0693-4902-acc7-8dcacb481126 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:03.845168 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:04:03.875197 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3e9d0aac-0693-4902-acc7-8dcacb481126 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6555c1eb8879440e8096459456dd4ebb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:03.884819 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3e9d0aac-0693-4902-acc7-8dcacb481126 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6555c1eb8879440e8096459456dd4ebb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:03.896619 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3e9d0aac-0693-4902-acc7-8dcacb481126 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6555c1eb8879440e8096459456dd4ebb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:03.897353 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3e9d0aac-0693-4902-acc7-8dcacb481126 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6555c1eb8879440e8096459456dd4ebb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:03.897981 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3e9d0aac-0693-4902-acc7-8dcacb481126 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6555c1eb8879440e8096459456dd4ebb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:03.918631 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1309/2619] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:04:03 2022] DELETE /identity/v3/users/6555c1eb8879440e8096459456dd4ebb => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:04:04.157892 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da814d-7fe7-4c6a-a847-073c2e4f2a56 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:04.159884 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d2da814d-7fe7-4c6a-a847-073c2e4f2a56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:04:04.176447 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da814d-7fe7-4c6a-a847-073c2e4f2a56 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:04.244642 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01c91d0-f157-4924-b2e3-991d7d039f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:04.246052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01c91d0-f157-4924-b2e3-991d7d039f22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:04.246052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01c91d0-f157-4924-b2e3-991d7d039f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:04.246052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01c91d0-f157-4924-b2e3-991d7d039f22 admin admin] PATH_INFO: `/v3/projects/d0eb535f5d514a8cb0d79b8e757fec4c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:04.247252 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01c91d0-f157-4924-b2e3-991d7d039f22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0eb535f5d514a8cb0d79b8e757fec4c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:04.249410 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01c91d0-f157-4924-b2e3-991d7d039f22 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:04.274938 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:04:04.312771 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f01c91d0-f157-4924-b2e3-991d7d039f22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0eb535f5d514a8cb0d79b8e757fec4c', 'request_id': 'req-f01c91d0-f157-4924-b2e3-991d7d039f22'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:04.324379 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f01c91d0-f157-4924-b2e3-991d7d039f22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0eb535f5d514a8cb0d79b8e757fec4c', 'request_id': 'req-f01c91d0-f157-4924-b2e3-991d7d039f22'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:04.331419 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f01c91d0-f157-4924-b2e3-991d7d039f22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0eb535f5d514a8cb0d79b8e757fec4c', 'request_id': 'req-f01c91d0-f157-4924-b2e3-991d7d039f22'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:04.343211 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f01c91d0-f157-4924-b2e3-991d7d039f22 admin admin] The token cache is being invalidate because project d0eb535f5d514a8cb0d79b8e757fec4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:04:04.343474 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f01c91d0-f157-4924-b2e3-991d7d039f22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:04.344181 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f01c91d0-f157-4924-b2e3-991d7d039f22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:04.345827 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1311/2620] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:04:04 2022] DELETE /identity/v3/projects/d0eb535f5d514a8cb0d79b8e757fec4c => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:04:04.351049 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853bb6b-58fc-4388-bf37-d5175b7130bb None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:04.351847 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853bb6b-58fc-4388-bf37-d5175b7130bb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:04.351847 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853bb6b-58fc-4388-bf37-d5175b7130bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:04.374059 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3853bb6b-58fc-4388-bf37-d5175b7130bb None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:04:04.375306 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3853bb6b-58fc-4388-bf37-d5175b7130bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:04:04.451754 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1310/2621] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:04:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 20:04:04.461088 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6010bc-f34c-49c9-8761-4b21ed295efb None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:04.461387 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6010bc-f34c-49c9-8761-4b21ed295efb None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:04.461651 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6010bc-f34c-49c9-8761-4b21ed295efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:04.485630 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0b6010bc-f34c-49c9-8761-4b21ed295efb None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:04:04.487501 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0b6010bc-f34c-49c9-8761-4b21ed295efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:04:04.497418 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1312/2622] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:04:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:04:04.502090 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853bb6b-58fc-4388-bf37-d5175b7130bb None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:04.512580 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853bb6b-58fc-4388-bf37-d5175b7130bb None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:04.518342 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7eda4f-b819-4590-8c13-2d31c1e94e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:04.519064 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7eda4f-b819-4590-8c13-2d31c1e94e1d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:04.519348 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7eda4f-b819-4590-8c13-2d31c1e94e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:04.519652 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7eda4f-b819-4590-8c13-2d31c1e94e1d admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:04.520387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7eda4f-b819-4590-8c13-2d31c1e94e1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 20:04:04.520697 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7eda4f-b819-4590-8c13-2d31c1e94e1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:04.522339 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7eda4f-b819-4590-8c13-2d31c1e94e1d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:04.536710 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1311/2623] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 20:04:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 20:04:04.541482 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6010bc-f34c-49c9-8761-4b21ed295efb None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:04.552371 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6010bc-f34c-49c9-8761-4b21ed295efb None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:04.557373 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad4ff46-fa82-4ef2-87af-180a7a9d0eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:04.558047 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad4ff46-fa82-4ef2-87af-180a7a9d0eac admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:04.558392 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad4ff46-fa82-4ef2-87af-180a7a9d0eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:04.558682 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad4ff46-fa82-4ef2-87af-180a7a9d0eac admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:04.559422 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad4ff46-fa82-4ef2-87af-180a7a9d0eac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:04.561431 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad4ff46-fa82-4ef2-87af-180a7a9d0eac admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:04.598486 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1313/2624] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:04:04 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 20:04:04.601533 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7eda4f-b819-4590-8c13-2d31c1e94e1d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:04.614047 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7eda4f-b819-4590-8c13-2d31c1e94e1d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:04.680997 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccceb122-a762-4ba8-a66f-200e9611e1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:04.681677 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccceb122-a762-4ba8-a66f-200e9611e1af admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:04.681989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccceb122-a762-4ba8-a66f-200e9611e1af admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:04.682269 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccceb122-a762-4ba8-a66f-200e9611e1af admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:04.682958 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccceb122-a762-4ba8-a66f-200e9611e1af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:04.684942 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccceb122-a762-4ba8-a66f-200e9611e1af admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:04.709023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1312/2625] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:04:04 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:04:04.713072 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad4ff46-fa82-4ef2-87af-180a7a9d0eac admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:04.727147 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad4ff46-fa82-4ef2-87af-180a7a9d0eac admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:04.734499 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dff83e-cc25-4201-b107-ed76f0610ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:04.735247 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dff83e-cc25-4201-b107-ed76f0610ce6 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:04.735687 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dff83e-cc25-4201-b107-ed76f0610ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:04.736713 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dff83e-cc25-4201-b107-ed76f0610ce6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:04.737713 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dff83e-cc25-4201-b107-ed76f0610ce6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:04.739941 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dff83e-cc25-4201-b107-ed76f0610ce6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:04.756327 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1314/2626] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 20:04:04.761044 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccceb122-a762-4ba8-a66f-200e9611e1af admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:04.773696 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccceb122-a762-4ba8-a66f-200e9611e1af admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:04.785914 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1328950-edc4-4d4d-a948-60bbe5a523ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:04.786641 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1328950-edc4-4d4d-a948-60bbe5a523ee admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:04.786930 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1328950-edc4-4d4d-a948-60bbe5a523ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:04.787350 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1328950-edc4-4d4d-a948-60bbe5a523ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:04.788167 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1328950-edc4-4d4d-a948-60bbe5a523ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:04.789886 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1328950-edc4-4d4d-a948-60bbe5a523ee admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:04.803862 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1313/2627] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:04:04.811973 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dff83e-cc25-4201-b107-ed76f0610ce6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:04.828309 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dff83e-cc25-4201-b107-ed76f0610ce6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:04.834752 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d032e09e-186e-45c1-bda6-4ae9da44a460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:04.835799 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032e09e-186e-45c1-bda6-4ae9da44a460 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:04.836104 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032e09e-186e-45c1-bda6-4ae9da44a460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:04.836390 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032e09e-186e-45c1-bda6-4ae9da44a460 admin admin] PATH_INFO: `/v3/projects/bb3183637ebd44288b6a4ae20bbfeb91/users/46de460c7ffb495d9880e09b33b4fd23/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:04.854752 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d032e09e-186e-45c1-bda6-4ae9da44a460 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb3183637ebd44288b6a4ae20bbfeb91, user_id=46de460c7ffb495d9880e09b33b4fd23, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:04.857604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d032e09e-186e-45c1-bda6-4ae9da44a460 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:04.872833 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1315/2628] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:04:04 2022] PUT /identity/v3/projects/bb3183637ebd44288b6a4ae20bbfeb91/users/46de460c7ffb495d9880e09b33b4fd23/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:04:04.887043 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e57a25-9a8d-4570-80c6-065c2dbe9356 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:04.887789 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e57a25-9a8d-4570-80c6-065c2dbe9356 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:04.888221 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e57a25-9a8d-4570-80c6-065c2dbe9356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:04.900092 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:04:04.924581 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-34e57a25-9a8d-4570-80c6-065c2dbe9356 None None] MFA Rules not processed for user `46de460c7ffb495d9880e09b33b4fd23`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:04:04.926486 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-34e57a25-9a8d-4570-80c6-065c2dbe9356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:04:05.006461 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1314/2629] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:04:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 20:04:05.010923 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d032e09e-186e-45c1-bda6-4ae9da44a460 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:05.078853 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d032e09e-186e-45c1-bda6-4ae9da44a460 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:05.084495 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91171ca2-4022-4fab-8f0a-87770db1b948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:05.085174 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91171ca2-4022-4fab-8f0a-87770db1b948 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:05.085464 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91171ca2-4022-4fab-8f0a-87770db1b948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:05.085750 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91171ca2-4022-4fab-8f0a-87770db1b948 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:05.086451 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91171ca2-4022-4fab-8f0a-87770db1b948 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:05.088415 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91171ca2-4022-4fab-8f0a-87770db1b948 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:05.124119 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1316/2630] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:04:05 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:04:05.128634 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e57a25-9a8d-4570-80c6-065c2dbe9356 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:05.140500 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e57a25-9a8d-4570-80c6-065c2dbe9356 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:05.200763 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3f13e9-1327-46de-85c6-5982ab8902ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:05.201442 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f13e9-1327-46de-85c6-5982ab8902ba admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:05.201733 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f13e9-1327-46de-85c6-5982ab8902ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:05.202012 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f13e9-1327-46de-85c6-5982ab8902ba admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:05.202755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3f13e9-1327-46de-85c6-5982ab8902ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:05.205947 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3f13e9-1327-46de-85c6-5982ab8902ba admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:05.228852 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1315/2631] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:04:05 2022] POST /identity/v3/users => generated 298 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 20:04:05.233057 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91171ca2-4022-4fab-8f0a-87770db1b948 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:05.247720 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91171ca2-4022-4fab-8f0a-87770db1b948 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:05.255206 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f66d21-3d76-4a81-9561-919151316981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:05.255932 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f66d21-3d76-4a81-9561-919151316981 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:05.256221 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f66d21-3d76-4a81-9561-919151316981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:05.256557 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f66d21-3d76-4a81-9561-919151316981 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:05.257253 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f66d21-3d76-4a81-9561-919151316981 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:05.259135 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f66d21-3d76-4a81-9561-919151316981 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:05.271761 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1317/2632] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:04:05 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:04:05.276075 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3f13e9-1327-46de-85c6-5982ab8902ba admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:05.289853 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3f13e9-1327-46de-85c6-5982ab8902ba admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:05.295929 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e31f3d-f462-4780-b821-8f10d24a086e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:05.297033 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e31f3d-f462-4780-b821-8f10d24a086e admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:05.297309 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e31f3d-f462-4780-b821-8f10d24a086e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:05.297637 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e31f3d-f462-4780-b821-8f10d24a086e admin admin] PATH_INFO: `/v3/projects/f7554009f3ee4131919aee8185c7db8c/users/817119835e9c4398879542a1ee60dc3d/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:05.319472 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e31f3d-f462-4780-b821-8f10d24a086e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7554009f3ee4131919aee8185c7db8c, user_id=817119835e9c4398879542a1ee60dc3d, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:05.323349 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e31f3d-f462-4780-b821-8f10d24a086e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:05.340431 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1316/2633] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:04:05 2022] PUT /identity/v3/projects/f7554009f3ee4131919aee8185c7db8c/users/817119835e9c4398879542a1ee60dc3d/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:04:05.344721 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f66d21-3d76-4a81-9561-919151316981 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:05.356686 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f66d21-3d76-4a81-9561-919151316981 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:05.384653 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:04:05.445739 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea54a1df-531a-4e2e-8367-a8ddd18baade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:05.446460 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54a1df-531a-4e2e-8367-a8ddd18baade admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:05.446750 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54a1df-531a-4e2e-8367-a8ddd18baade admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:05.447084 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54a1df-531a-4e2e-8367-a8ddd18baade admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:05.447857 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea54a1df-531a-4e2e-8367-a8ddd18baade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:05.449536 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea54a1df-531a-4e2e-8367-a8ddd18baade admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:05.460979 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1318/2634] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:04:05 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 20:04:05.465272 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e31f3d-f462-4780-b821-8f10d24a086e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:05.477417 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e31f3d-f462-4780-b821-8f10d24a086e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:05.482540 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d1477-ea4c-4764-86af-b3e6dde2c764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:05.483476 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d1477-ea4c-4764-86af-b3e6dde2c764 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:05.483797 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d1477-ea4c-4764-86af-b3e6dde2c764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:05.484073 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d1477-ea4c-4764-86af-b3e6dde2c764 admin admin] PATH_INFO: `/v3/projects/f7554009f3ee4131919aee8185c7db8c/users/817119835e9c4398879542a1ee60dc3d/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:05.486793 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d1477-ea4c-4764-86af-b3e6dde2c764 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7554009f3ee4131919aee8185c7db8c, user_id=817119835e9c4398879542a1ee60dc3d, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:05.489538 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d1477-ea4c-4764-86af-b3e6dde2c764 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:05.503139 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1317/2635] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:04:05 2022] PUT /identity/v3/projects/f7554009f3ee4131919aee8185c7db8c/users/817119835e9c4398879542a1ee60dc3d/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 20:04:05.514869 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcaf735-4752-449e-b6c7-c4e09a265011 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:05.515415 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcaf735-4752-449e-b6c7-c4e09a265011 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:05.515674 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcaf735-4752-449e-b6c7-c4e09a265011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:05.546905 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1fcaf735-4752-449e-b6c7-c4e09a265011 None None] MFA Rules not processed for user `817119835e9c4398879542a1ee60dc3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:04:05.548298 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1fcaf735-4752-449e-b6c7-c4e09a265011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:04:05.622890 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1319/2636] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:04:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:04:05.637595 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d1477-ea4c-4764-86af-b3e6dde2c764 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:05.639318 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-692d1477-ea4c-4764-86af-b3e6dde2c764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:04:05.698303 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:04:05.705906 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d1477-ea4c-4764-86af-b3e6dde2c764 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:05.710020 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec97c3-75f4-4487-8641-7987fe65bc6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:05.710713 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec97c3-75f4-4487-8641-7987fe65bc6e service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:05.711001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec97c3-75f4-4487-8641-7987fe65bc6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:05.711279 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec97c3-75f4-4487-8641-7987fe65bc6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:05.778554 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec97c3-75f4-4487-8641-7987fe65bc6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:05.779947 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec97c3-75f4-4487-8641-7987fe65bc6e service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:05.791369 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1318/2637] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:04:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:04:10.317046 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcaf735-4752-449e-b6c7-c4e09a265011 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:10.332512 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcaf735-4752-449e-b6c7-c4e09a265011 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:10.338669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741b9c7-3abc-4967-b855-d73b6e7626e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:10.339343 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741b9c7-3abc-4967-b855-d73b6e7626e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:10.339680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741b9c7-3abc-4967-b855-d73b6e7626e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:10.339967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741b9c7-3abc-4967-b855-d73b6e7626e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:10.353386 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741b9c7-3abc-4967-b855-d73b6e7626e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:10.354862 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741b9c7-3abc-4967-b855-d73b6e7626e6 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:10.365233 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1320/2638] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:04:10 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:04:15.347902 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec97c3-75f4-4487-8641-7987fe65bc6e service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:15.419837 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec97c3-75f4-4487-8641-7987fe65bc6e service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:15.425589 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a01c04-07a4-41a5-9e95-1bce8fede7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:15.426408 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a01c04-07a4-41a5-9e95-1bce8fede7c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:15.426694 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a01c04-07a4-41a5-9e95-1bce8fede7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:15.426974 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a01c04-07a4-41a5-9e95-1bce8fede7c3 admin admin] PATH_INFO: `/v3/users/46de460c7ffb495d9880e09b33b4fd23` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:15.428259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a01c04-07a4-41a5-9e95-1bce8fede7c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46de460c7ffb495d9880e09b33b4fd23)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:15.430223 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a01c04-07a4-41a5-9e95-1bce8fede7c3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:15.484773 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-55a01c04-07a4-41a5-9e95-1bce8fede7c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46de460c7ffb495d9880e09b33b4fd23'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:15.492334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-55a01c04-07a4-41a5-9e95-1bce8fede7c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46de460c7ffb495d9880e09b33b4fd23'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:15.501106 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-55a01c04-07a4-41a5-9e95-1bce8fede7c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46de460c7ffb495d9880e09b33b4fd23'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:15.502033 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-55a01c04-07a4-41a5-9e95-1bce8fede7c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46de460c7ffb495d9880e09b33b4fd23'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:15.502666 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-55a01c04-07a4-41a5-9e95-1bce8fede7c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46de460c7ffb495d9880e09b33b4fd23'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:15.524313 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1319/2639] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:04:15 2022] DELETE /identity/v3/users/46de460c7ffb495d9880e09b33b4fd23 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:04:15.529189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075d084-76ab-420b-a188-df2ff8c59afb None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:15.529773 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075d084-76ab-420b-a188-df2ff8c59afb None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:15.529773 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075d084-76ab-420b-a188-df2ff8c59afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:15.550586 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5075d084-76ab-420b-a188-df2ff8c59afb None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:04:15.551934 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5075d084-76ab-420b-a188-df2ff8c59afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:04:15.629145 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1321/2640] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:04:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:04:15.639758 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a01c04-07a4-41a5-9e95-1bce8fede7c3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:15.641236 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-55a01c04-07a4-41a5-9e95-1bce8fede7c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:04:15.666533 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:04:15.708891 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a01c04-07a4-41a5-9e95-1bce8fede7c3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:15.712743 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f1651-edd5-41a8-aaa2-ed7ca42ca20d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:15.713402 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f1651-edd5-41a8-aaa2-ed7ca42ca20d service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:15.713693 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f1651-edd5-41a8-aaa2-ed7ca42ca20d service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:15.713981 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f1651-edd5-41a8-aaa2-ed7ca42ca20d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:15.724512 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949f1651-edd5-41a8-aaa2-ed7ca42ca20d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:15.726000 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949f1651-edd5-41a8-aaa2-ed7ca42ca20d service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:15.735652 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1320/2641] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:04:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 20:04:16.036334 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5075d084-76ab-420b-a188-df2ff8c59afb None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:16.037784 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5075d084-76ab-420b-a188-df2ff8c59afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:04:16.053548 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5075d084-76ab-420b-a188-df2ff8c59afb None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:16.062968 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f710dc2-6c1a-4e27-b3e5-234a4993859a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:16.063675 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f710dc2-6c1a-4e27-b3e5-234a4993859a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:16.065302 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f710dc2-6c1a-4e27-b3e5-234a4993859a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:16.065932 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f710dc2-6c1a-4e27-b3e5-234a4993859a admin admin] PATH_INFO: `/v3/projects/bb3183637ebd44288b6a4ae20bbfeb91` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:16.067832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f710dc2-6c1a-4e27-b3e5-234a4993859a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb3183637ebd44288b6a4ae20bbfeb91)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:16.070954 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f710dc2-6c1a-4e27-b3e5-234a4993859a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:16.106247 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:04:16.134285 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4f710dc2-6c1a-4e27-b3e5-234a4993859a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb3183637ebd44288b6a4ae20bbfeb91', 'request_id': 'req-4f710dc2-6c1a-4e27-b3e5-234a4993859a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:16.142940 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4f710dc2-6c1a-4e27-b3e5-234a4993859a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb3183637ebd44288b6a4ae20bbfeb91', 'request_id': 'req-4f710dc2-6c1a-4e27-b3e5-234a4993859a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:16.149708 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4f710dc2-6c1a-4e27-b3e5-234a4993859a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb3183637ebd44288b6a4ae20bbfeb91', 'request_id': 'req-4f710dc2-6c1a-4e27-b3e5-234a4993859a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:16.163620 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4f710dc2-6c1a-4e27-b3e5-234a4993859a admin admin] The token cache is being invalidate because project bb3183637ebd44288b6a4ae20bbfeb91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:04:16.164085 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4f710dc2-6c1a-4e27-b3e5-234a4993859a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:16.164895 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4f710dc2-6c1a-4e27-b3e5-234a4993859a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:16.166607 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1322/2642] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:04:16 2022] DELETE /identity/v3/projects/bb3183637ebd44288b6a4ae20bbfeb91 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:04:16.170961 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f1651-edd5-41a8-aaa2-ed7ca42ca20d service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:16.182351 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f1651-edd5-41a8-aaa2-ed7ca42ca20d service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:16.243754 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dcb410-04f0-444e-b119-4337fc322d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:16.244551 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dcb410-04f0-444e-b119-4337fc322d2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:16.244842 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dcb410-04f0-444e-b119-4337fc322d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:16.245127 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dcb410-04f0-444e-b119-4337fc322d2c admin admin] PATH_INFO: `/v3/users/817119835e9c4398879542a1ee60dc3d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:16.246361 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dcb410-04f0-444e-b119-4337fc322d2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=817119835e9c4398879542a1ee60dc3d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:16.248371 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dcb410-04f0-444e-b119-4337fc322d2c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:16.307121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e0dcb410-04f0-444e-b119-4337fc322d2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '817119835e9c4398879542a1ee60dc3d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:16.316327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e0dcb410-04f0-444e-b119-4337fc322d2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '817119835e9c4398879542a1ee60dc3d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:16.325521 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e0dcb410-04f0-444e-b119-4337fc322d2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '817119835e9c4398879542a1ee60dc3d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:16.326365 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e0dcb410-04f0-444e-b119-4337fc322d2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '817119835e9c4398879542a1ee60dc3d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:16.327263 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e0dcb410-04f0-444e-b119-4337fc322d2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '817119835e9c4398879542a1ee60dc3d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:16.351959 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1321/2643] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:04:16 2022] DELETE /identity/v3/users/817119835e9c4398879542a1ee60dc3d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:04:16.651560 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f710dc2-6c1a-4e27-b3e5-234a4993859a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:16.653446 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4f710dc2-6c1a-4e27-b3e5-234a4993859a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:04:16.673311 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f710dc2-6c1a-4e27-b3e5-234a4993859a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:16.738313 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95eac5-630c-45be-8f90-f1e0078ddb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:16.739714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95eac5-630c-45be-8f90-f1e0078ddb66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:16.740145 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95eac5-630c-45be-8f90-f1e0078ddb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:16.740564 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95eac5-630c-45be-8f90-f1e0078ddb66 admin admin] PATH_INFO: `/v3/projects/f7554009f3ee4131919aee8185c7db8c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:16.742674 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb95eac5-630c-45be-8f90-f1e0078ddb66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7554009f3ee4131919aee8185c7db8c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:16.745874 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb95eac5-630c-45be-8f90-f1e0078ddb66 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:16.814854 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bb95eac5-630c-45be-8f90-f1e0078ddb66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7554009f3ee4131919aee8185c7db8c', 'request_id': 'req-bb95eac5-630c-45be-8f90-f1e0078ddb66'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:16.826655 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bb95eac5-630c-45be-8f90-f1e0078ddb66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7554009f3ee4131919aee8185c7db8c', 'request_id': 'req-bb95eac5-630c-45be-8f90-f1e0078ddb66'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:16.836603 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bb95eac5-630c-45be-8f90-f1e0078ddb66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7554009f3ee4131919aee8185c7db8c', 'request_id': 'req-bb95eac5-630c-45be-8f90-f1e0078ddb66'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:16.854409 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bb95eac5-630c-45be-8f90-f1e0078ddb66 admin admin] The token cache is being invalidate because project f7554009f3ee4131919aee8185c7db8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:04:16.854703 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bb95eac5-630c-45be-8f90-f1e0078ddb66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:16.856418 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bb95eac5-630c-45be-8f90-f1e0078ddb66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:16.858948 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1323/2644] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:04:16 2022] DELETE /identity/v3/projects/f7554009f3ee4131919aee8185c7db8c => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:04:16.866708 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54e4f8-e0af-419a-874e-2f77e9f08005 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:16.867282 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54e4f8-e0af-419a-874e-2f77e9f08005 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:16.867739 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54e4f8-e0af-419a-874e-2f77e9f08005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:16.897109 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-af54e4f8-e0af-419a-874e-2f77e9f08005 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:04:16.899079 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-af54e4f8-e0af-419a-874e-2f77e9f08005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:04:16.962320 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:04:16.989735 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1322/2645] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:04:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:04:16.998755 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffdd77-774e-4f61-a37b-6dade1c1ba89 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:16.999218 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffdd77-774e-4f61-a37b-6dade1c1ba89 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:16.999632 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffdd77-774e-4f61-a37b-6dade1c1ba89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:17.029659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-67ffdd77-774e-4f61-a37b-6dade1c1ba89 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:04:17.030921 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-67ffdd77-774e-4f61-a37b-6dade1c1ba89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:04:17.046592 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1324/2646] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:04:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:04:17.051472 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af54e4f8-e0af-419a-874e-2f77e9f08005 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:17.062103 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af54e4f8-e0af-419a-874e-2f77e9f08005 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:17.068687 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2941dd0a-bfab-4902-b775-b0a496a0f3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:17.069823 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941dd0a-bfab-4902-b775-b0a496a0f3ca admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:17.070493 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941dd0a-bfab-4902-b775-b0a496a0f3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:17.070752 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941dd0a-bfab-4902-b775-b0a496a0f3ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:17.071816 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2941dd0a-bfab-4902-b775-b0a496a0f3ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 20:04:17.072379 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2941dd0a-bfab-4902-b775-b0a496a0f3ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:17.075154 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2941dd0a-bfab-4902-b775-b0a496a0f3ca admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:17.097731 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1323/2647] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 20:04:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 20:04:17.104722 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffdd77-774e-4f61-a37b-6dade1c1ba89 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:17.121278 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffdd77-774e-4f61-a37b-6dade1c1ba89 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:17.129257 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7335b44f-bad1-4099-b721-c2e1519a660b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:17.130333 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335b44f-bad1-4099-b721-c2e1519a660b admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:17.130809 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335b44f-bad1-4099-b721-c2e1519a660b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:17.131268 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335b44f-bad1-4099-b721-c2e1519a660b admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:17.132499 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7335b44f-bad1-4099-b721-c2e1519a660b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:17.135984 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7335b44f-bad1-4099-b721-c2e1519a660b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:17.172246 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1325/2648] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:04:17 2022] POST /identity/v3/projects => generated 359 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:04:17.182228 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2941dd0a-bfab-4902-b775-b0a496a0f3ca admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:17.196321 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2941dd0a-bfab-4902-b775-b0a496a0f3ca admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:17.288062 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd5d38e-f1b5-4c44-a31a-3ae779140101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:17.288768 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd5d38e-f1b5-4c44-a31a-3ae779140101 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:17.289053 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd5d38e-f1b5-4c44-a31a-3ae779140101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:17.289333 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd5d38e-f1b5-4c44-a31a-3ae779140101 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:17.290038 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd5d38e-f1b5-4c44-a31a-3ae779140101 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:17.292128 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd5d38e-f1b5-4c44-a31a-3ae779140101 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:17.317127 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1324/2649] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:04:17 2022] POST /identity/v3/users => generated 280 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:04:17.321920 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7335b44f-bad1-4099-b721-c2e1519a660b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:17.336067 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7335b44f-bad1-4099-b721-c2e1519a660b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:17.344024 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b29c54-177a-4064-ba41-75df1e531780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:17.345604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b29c54-177a-4064-ba41-75df1e531780 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:17.346030 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b29c54-177a-4064-ba41-75df1e531780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:17.346463 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b29c54-177a-4064-ba41-75df1e531780 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:17.347588 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b29c54-177a-4064-ba41-75df1e531780 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:17.350686 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b29c54-177a-4064-ba41-75df1e531780 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:17.358067 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:04:17.364967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1326/2650] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:04:17.369360 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd5d38e-f1b5-4c44-a31a-3ae779140101 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:17.379691 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd5d38e-f1b5-4c44-a31a-3ae779140101 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:17.384532 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756b600-13e1-4970-950f-564801e042e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:17.385134 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756b600-13e1-4970-950f-564801e042e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:17.385421 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756b600-13e1-4970-950f-564801e042e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:17.385705 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756b600-13e1-4970-950f-564801e042e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:17.386493 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4756b600-13e1-4970-950f-564801e042e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:17.388132 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4756b600-13e1-4970-950f-564801e042e4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:17.399960 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1325/2651] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:04:17.404735 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b29c54-177a-4064-ba41-75df1e531780 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:17.416365 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b29c54-177a-4064-ba41-75df1e531780 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:17.421160 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e7fa9d-3706-4f0b-8c73-c478402f5398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:17.422053 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e7fa9d-3706-4f0b-8c73-c478402f5398 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:17.422405 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e7fa9d-3706-4f0b-8c73-c478402f5398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:17.422673 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e7fa9d-3706-4f0b-8c73-c478402f5398 admin admin] PATH_INFO: `/v3/projects/0cdabf42475e47018e1daa4aec96423d/users/f66213f1a954478890f97d75d24f3d90/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:17.440817 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e7fa9d-3706-4f0b-8c73-c478402f5398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cdabf42475e47018e1daa4aec96423d, user_id=f66213f1a954478890f97d75d24f3d90, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:17.443774 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e7fa9d-3706-4f0b-8c73-c478402f5398 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:17.460453 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1327/2652] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:04:17 2022] PUT /identity/v3/projects/0cdabf42475e47018e1daa4aec96423d/users/f66213f1a954478890f97d75d24f3d90/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:04:17.476026 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1981eeb-ba01-4bef-b4f4-0983de54be86 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:17.476314 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1981eeb-ba01-4bef-b4f4-0983de54be86 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:17.476590 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1981eeb-ba01-4bef-b4f4-0983de54be86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:17.511847 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f1981eeb-ba01-4bef-b4f4-0983de54be86 None None] MFA Rules not processed for user `f66213f1a954478890f97d75d24f3d90`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:04:17.513255 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f1981eeb-ba01-4bef-b4f4-0983de54be86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:04:17.585262 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1326/2653] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:04:17 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 20:04:17.619592 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e7fa9d-3706-4f0b-8c73-c478402f5398 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:17.621497 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c9e7fa9d-3706-4f0b-8c73-c478402f5398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:04:17.695471 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e7fa9d-3706-4f0b-8c73-c478402f5398 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:17.699442 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e8ad9-6db3-4bd4-9326-39b3e7f3b5e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:17.700122 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e8ad9-6db3-4bd4-9326-39b3e7f3b5e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:17.700411 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e8ad9-6db3-4bd4-9326-39b3e7f3b5e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:17.700695 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e8ad9-6db3-4bd4-9326-39b3e7f3b5e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:17.710423 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e8ad9-6db3-4bd4-9326-39b3e7f3b5e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:17.711931 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e8ad9-6db3-4bd4-9326-39b3e7f3b5e0 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:17.721798 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1328/2654] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:04:17 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:04:17.738004 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1981eeb-ba01-4bef-b4f4-0983de54be86 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:17.786799 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:04:17.820897 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1981eeb-ba01-4bef-b4f4-0983de54be86 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:17.825561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b683926-d008-4b52-9031-51e879cbcaf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:17.826433 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b683926-d008-4b52-9031-51e879cbcaf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:17.826716 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b683926-d008-4b52-9031-51e879cbcaf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:17.826991 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b683926-d008-4b52-9031-51e879cbcaf5 admin admin] PATH_INFO: `/v3/users/f66213f1a954478890f97d75d24f3d90` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:17.828228 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b683926-d008-4b52-9031-51e879cbcaf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f66213f1a954478890f97d75d24f3d90)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:17.831274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b683926-d008-4b52-9031-51e879cbcaf5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:17.884409 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2b683926-d008-4b52-9031-51e879cbcaf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f66213f1a954478890f97d75d24f3d90'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:17.891391 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2b683926-d008-4b52-9031-51e879cbcaf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f66213f1a954478890f97d75d24f3d90'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:17.900030 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2b683926-d008-4b52-9031-51e879cbcaf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f66213f1a954478890f97d75d24f3d90'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:17.900675 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2b683926-d008-4b52-9031-51e879cbcaf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f66213f1a954478890f97d75d24f3d90'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:17.901335 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2b683926-d008-4b52-9031-51e879cbcaf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f66213f1a954478890f97d75d24f3d90'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:17.920141 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1327/2655] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:04:17 2022] DELETE /identity/v3/users/f66213f1a954478890f97d75d24f3d90 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:04:17.925384 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a57c42-e573-4d7f-aee9-9bed313d4818 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:17.925676 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a57c42-e573-4d7f-aee9-9bed313d4818 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:17.925959 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a57c42-e573-4d7f-aee9-9bed313d4818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:17.945934 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b3a57c42-e573-4d7f-aee9-9bed313d4818 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:04:17.947160 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b3a57c42-e573-4d7f-aee9-9bed313d4818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:04:18.022090 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1329/2656] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:04:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:04:18.032408 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b683926-d008-4b52-9031-51e879cbcaf5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:18.034514 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2b683926-d008-4b52-9031-51e879cbcaf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:04:18.099459 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b683926-d008-4b52-9031-51e879cbcaf5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:18.103915 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161b99c-bafd-4569-b66d-ee10fd56cfa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:18.104579 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161b99c-bafd-4569-b66d-ee10fd56cfa2 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:18.104869 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161b99c-bafd-4569-b66d-ee10fd56cfa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:18.105165 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161b99c-bafd-4569-b66d-ee10fd56cfa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:18.115838 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161b99c-bafd-4569-b66d-ee10fd56cfa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:18.117693 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161b99c-bafd-4569-b66d-ee10fd56cfa2 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:18.129078 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1328/2657] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:04:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:04:18.432401 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a57c42-e573-4d7f-aee9-9bed313d4818 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:18.433922 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b3a57c42-e573-4d7f-aee9-9bed313d4818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:04:18.447235 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a57c42-e573-4d7f-aee9-9bed313d4818 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:18.452136 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674d7e44-738e-4f7e-9034-8220124dc575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:18.453002 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d7e44-738e-4f7e-9034-8220124dc575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:18.453440 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d7e44-738e-4f7e-9034-8220124dc575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:18.453742 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d7e44-738e-4f7e-9034-8220124dc575 admin admin] PATH_INFO: `/v3/projects/0cdabf42475e47018e1daa4aec96423d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:18.455725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674d7e44-738e-4f7e-9034-8220124dc575 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cdabf42475e47018e1daa4aec96423d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:18.457787 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674d7e44-738e-4f7e-9034-8220124dc575 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:18.506704 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-674d7e44-738e-4f7e-9034-8220124dc575 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cdabf42475e47018e1daa4aec96423d', 'request_id': 'req-674d7e44-738e-4f7e-9034-8220124dc575'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:18.514968 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-674d7e44-738e-4f7e-9034-8220124dc575 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cdabf42475e47018e1daa4aec96423d', 'request_id': 'req-674d7e44-738e-4f7e-9034-8220124dc575'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:18.521836 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-674d7e44-738e-4f7e-9034-8220124dc575 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cdabf42475e47018e1daa4aec96423d', 'request_id': 'req-674d7e44-738e-4f7e-9034-8220124dc575'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:18.522982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:04:18.536370 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-674d7e44-738e-4f7e-9034-8220124dc575 admin admin] The token cache is being invalidate because project 0cdabf42475e47018e1daa4aec96423d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:04:18.536655 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-674d7e44-738e-4f7e-9034-8220124dc575 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:18.537298 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-674d7e44-738e-4f7e-9034-8220124dc575 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:04:18.538828 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1330/2658] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:04:18 2022] DELETE /identity/v3/projects/0cdabf42475e47018e1daa4aec96423d => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 20:04:18.543873 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3fa76-9497-4805-bfb4-b67ea483df08 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:18.544392 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3fa76-9497-4805-bfb4-b67ea483df08 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:18.544500 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3fa76-9497-4805-bfb4-b67ea483df08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:18.565369 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-59d3fa76-9497-4805-bfb4-b67ea483df08 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:04:18.566604 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-59d3fa76-9497-4805-bfb4-b67ea483df08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:04:18.621312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:04:18.631794 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1329/2659] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:04:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:04:18.639484 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7bb0dd-dda0-429c-bae8-40eb0d3ece53 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:18.639900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7bb0dd-dda0-429c-bae8-40eb0d3ece53 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:18.640178 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7bb0dd-dda0-429c-bae8-40eb0d3ece53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:18.661505 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-fc7bb0dd-dda0-429c-bae8-40eb0d3ece53 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:04:18.662709 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fc7bb0dd-dda0-429c-bae8-40eb0d3ece53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:04:18.671292 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1331/2660] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:04:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 20:04:18.676542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d3fa76-9497-4805-bfb4-b67ea483df08 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:18.686551 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d3fa76-9497-4805-bfb4-b67ea483df08 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:18.692043 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f0da6-3a95-4cf1-85d1-528fc10012e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:18.692710 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f0da6-3a95-4cf1-85d1-528fc10012e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:18.692996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f0da6-3a95-4cf1-85d1-528fc10012e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:18.693277 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f0da6-3a95-4cf1-85d1-528fc10012e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:18.694012 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f0da6-3a95-4cf1-85d1-528fc10012e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 20:04:18.694319 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f0da6-3a95-4cf1-85d1-528fc10012e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:18.695971 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f0da6-3a95-4cf1-85d1-528fc10012e0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:18.709133 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1330/2661] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 20:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 20:04:18.713777 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7bb0dd-dda0-429c-bae8-40eb0d3ece53 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:18.723796 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7bb0dd-dda0-429c-bae8-40eb0d3ece53 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:18.728588 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7b772-374d-4209-92e0-1f3343a1547b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:18.729252 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7b772-374d-4209-92e0-1f3343a1547b admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:18.729537 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7b772-374d-4209-92e0-1f3343a1547b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:18.729810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7b772-374d-4209-92e0-1f3343a1547b admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:18.730497 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf7b772-374d-4209-92e0-1f3343a1547b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:18.732454 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf7b772-374d-4209-92e0-1f3343a1547b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:18.763853 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1332/2662] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:04:18 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:04:18.768679 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f0da6-3a95-4cf1-85d1-528fc10012e0 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:18.779973 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f0da6-3a95-4cf1-85d1-528fc10012e0 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:18.840810 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d1641d-5d15-4022-95dc-d6cd24fbd309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:18.841488 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d1641d-5d15-4022-95dc-d6cd24fbd309 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:18.841774 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d1641d-5d15-4022-95dc-d6cd24fbd309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:18.842056 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d1641d-5d15-4022-95dc-d6cd24fbd309 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:18.842744 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d1641d-5d15-4022-95dc-d6cd24fbd309 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:18.844721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d1641d-5d15-4022-95dc-d6cd24fbd309 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:18.868214 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1331/2663] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:04:18 2022] POST /identity/v3/users => generated 283 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:04:18.872647 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7b772-374d-4209-92e0-1f3343a1547b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:18.884917 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7b772-374d-4209-92e0-1f3343a1547b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:18.896224 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736804c-8584-42e8-acb9-1cd5021644e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:18.896893 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736804c-8584-42e8-acb9-1cd5021644e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:18.897148 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736804c-8584-42e8-acb9-1cd5021644e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:18.897428 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736804c-8584-42e8-acb9-1cd5021644e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:18.898206 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d736804c-8584-42e8-acb9-1cd5021644e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:18.899880 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d736804c-8584-42e8-acb9-1cd5021644e5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:18.912512 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1333/2664] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:04:18.917200 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d1641d-5d15-4022-95dc-d6cd24fbd309 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:18.929090 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d1641d-5d15-4022-95dc-d6cd24fbd309 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:18.934024 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734f0fe-20f2-4e32-8a1c-5de2674cf353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:18.934698 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734f0fe-20f2-4e32-8a1c-5de2674cf353 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:18.934995 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734f0fe-20f2-4e32-8a1c-5de2674cf353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:18.935277 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734f0fe-20f2-4e32-8a1c-5de2674cf353 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:18.936046 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1734f0fe-20f2-4e32-8a1c-5de2674cf353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:18.937655 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1734f0fe-20f2-4e32-8a1c-5de2674cf353 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:18.950324 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1332/2665] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:04:18.954928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736804c-8584-42e8-acb9-1cd5021644e5 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:18.965906 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736804c-8584-42e8-acb9-1cd5021644e5 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:18.971978 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773b3f1-756f-485a-8962-21d6fe21ccf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:18.973398 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773b3f1-756f-485a-8962-21d6fe21ccf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:18.973899 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773b3f1-756f-485a-8962-21d6fe21ccf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:18.974384 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773b3f1-756f-485a-8962-21d6fe21ccf3 admin admin] PATH_INFO: `/v3/projects/b41e1c619c834ee7bbc7c2c502927ff9/users/191be5b1ab65475188c0e0773e308862/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:19.001075 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773b3f1-756f-485a-8962-21d6fe21ccf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b41e1c619c834ee7bbc7c2c502927ff9, user_id=191be5b1ab65475188c0e0773e308862, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:19.006925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773b3f1-756f-485a-8962-21d6fe21ccf3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:19.025433 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1334/2666] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:04:18 2022] PUT /identity/v3/projects/b41e1c619c834ee7bbc7c2c502927ff9/users/191be5b1ab65475188c0e0773e308862/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:04:19.037248 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0f665-102b-4f12-8706-4ebf45d83b4a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:19.037540 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0f665-102b-4f12-8706-4ebf45d83b4a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:19.037820 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0f665-102b-4f12-8706-4ebf45d83b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:19.201307 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d0b0f665-102b-4f12-8706-4ebf45d83b4a None None] MFA Rules not processed for user `191be5b1ab65475188c0e0773e308862`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:04:19.202451 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d0b0f665-102b-4f12-8706-4ebf45d83b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:04:19.298346 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1333/2667] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:04:19 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:04:19.313987 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773b3f1-756f-485a-8962-21d6fe21ccf3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:04:19.315793 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3773b3f1-756f-485a-8962-21d6fe21ccf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:04:19.380161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773b3f1-756f-485a-8962-21d6fe21ccf3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:04:19.383886 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea522bbf-f5ad-4ae9-b148-4cf6104d107a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:04:19.384539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea522bbf-f5ad-4ae9-b148-4cf6104d107a service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:04:19.384830 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea522bbf-f5ad-4ae9-b148-4cf6104d107a service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:04:19.385113 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea522bbf-f5ad-4ae9-b148-4cf6104d107a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:04:19.394607 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea522bbf-f5ad-4ae9-b148-4cf6104d107a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:04:19.396104 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea522bbf-f5ad-4ae9-b148-4cf6104d107a service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:04:19.406063 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1335/2668] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:04:19 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:04:29.292416 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=72,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:04:29.401349 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=56,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:05:00.573390 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b0f665-102b-4f12-8706-4ebf45d83b4a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:05:00.644818 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b0f665-102b-4f12-8706-4ebf45d83b4a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:05:00.652209 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78fb691-4550-4282-9096-10cb30115c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:05:00.653112 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fb691-4550-4282-9096-10cb30115c6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:05:00.653395 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fb691-4550-4282-9096-10cb30115c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:05:00.655737 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fb691-4550-4282-9096-10cb30115c6c admin admin] PATH_INFO: `/v3/users/191be5b1ab65475188c0e0773e308862` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:05:00.655737 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78fb691-4550-4282-9096-10cb30115c6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=191be5b1ab65475188c0e0773e308862)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:05:00.657567 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78fb691-4550-4282-9096-10cb30115c6c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:05:00.714401 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a78fb691-4550-4282-9096-10cb30115c6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '191be5b1ab65475188c0e0773e308862'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:05:00.725553 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a78fb691-4550-4282-9096-10cb30115c6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '191be5b1ab65475188c0e0773e308862'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:05:00.737123 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a78fb691-4550-4282-9096-10cb30115c6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '191be5b1ab65475188c0e0773e308862'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:05:00.737775 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a78fb691-4550-4282-9096-10cb30115c6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '191be5b1ab65475188c0e0773e308862'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:05:00.738463 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a78fb691-4550-4282-9096-10cb30115c6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '191be5b1ab65475188c0e0773e308862'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:05:00.762602 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1334/2669] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:05:00 2022] DELETE /identity/v3/users/191be5b1ab65475188c0e0773e308862 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:05:00.768280 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7832e43-5480-47a1-b561-4466400976d5 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:05:00.768796 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7832e43-5480-47a1-b561-4466400976d5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:05:00.768996 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7832e43-5480-47a1-b561-4466400976d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:05:00.790545 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d7832e43-5480-47a1-b561-4466400976d5 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:05:00.791982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d7832e43-5480-47a1-b561-4466400976d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:05:00.858121 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1336/2670] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:05:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 20:05:00.869248 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78fb691-4550-4282-9096-10cb30115c6c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:05:00.870660 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a78fb691-4550-4282-9096-10cb30115c6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:05:00.949013 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78fb691-4550-4282-9096-10cb30115c6c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:05:00.952693 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f0bb1f-c05b-4501-8b83-fb63c7f2d8c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:05:00.953410 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0bb1f-c05b-4501-8b83-fb63c7f2d8c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:05:00.953855 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0bb1f-c05b-4501-8b83-fb63c7f2d8c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:05:00.954140 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0bb1f-c05b-4501-8b83-fb63c7f2d8c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:05:00.964337 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f0bb1f-c05b-4501-8b83-fb63c7f2d8c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:05:00.965783 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f0bb1f-c05b-4501-8b83-fb63c7f2d8c1 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:05:00.977304 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1335/2671] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:05:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:05:01.337950 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7832e43-5480-47a1-b561-4466400976d5 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:05:01.339677 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d7832e43-5480-47a1-b561-4466400976d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:05:01.352636 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7832e43-5480-47a1-b561-4466400976d5 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:05:01.358798 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f09b0-e813-48be-acd9-b18689261b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:05:01.360377 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f09b0-e813-48be-acd9-b18689261b8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:05:01.360663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f09b0-e813-48be-acd9-b18689261b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:05:01.360937 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f09b0-e813-48be-acd9-b18689261b8c admin admin] PATH_INFO: `/v3/projects/b41e1c619c834ee7bbc7c2c502927ff9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:05:01.362189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f09b0-e813-48be-acd9-b18689261b8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b41e1c619c834ee7bbc7c2c502927ff9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:05:01.364181 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f09b0-e813-48be-acd9-b18689261b8c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:05:01.415256 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-379f09b0-e813-48be-acd9-b18689261b8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b41e1c619c834ee7bbc7c2c502927ff9', 'request_id': 'req-379f09b0-e813-48be-acd9-b18689261b8c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:05:01.424035 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-379f09b0-e813-48be-acd9-b18689261b8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b41e1c619c834ee7bbc7c2c502927ff9', 'request_id': 'req-379f09b0-e813-48be-acd9-b18689261b8c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:05:01.430075 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-379f09b0-e813-48be-acd9-b18689261b8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b41e1c619c834ee7bbc7c2c502927ff9', 'request_id': 'req-379f09b0-e813-48be-acd9-b18689261b8c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:05:01.442216 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-379f09b0-e813-48be-acd9-b18689261b8c admin admin] The token cache is being invalidate because project b41e1c619c834ee7bbc7c2c502927ff9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:05:01.442511 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-379f09b0-e813-48be-acd9-b18689261b8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:05:01.443173 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-379f09b0-e813-48be-acd9-b18689261b8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:05:01.444748 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1337/2672] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:05:01 2022] DELETE /identity/v3/projects/b41e1c619c834ee7bbc7c2c502927ff9 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:05:01.892376 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68db0fe-3630-43a3-8b5d-a413fc311025 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:05:01.892885 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68db0fe-3630-43a3-8b5d-a413fc311025 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:05:01.893042 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68db0fe-3630-43a3-8b5d-a413fc311025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:05:01.924490 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e68db0fe-3630-43a3-8b5d-a413fc311025 None None] MFA Rules not processed for user `1c0772205ae24713b8901666a87018b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:05:01.929554 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e68db0fe-3630-43a3-8b5d-a413fc311025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:05:01.950113 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:05:02.005096 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1336/2673] 10.222.0.18 () {58 vars in 967 bytes} [Mon Jul 25 20:05:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:05:02.048770 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f09b0-e813-48be-acd9-b18689261b8c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:05:02.050286 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-379f09b0-e813-48be-acd9-b18689261b8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:05:02.145548 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f09b0-e813-48be-acd9-b18689261b8c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:05:02.151026 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9763125d-913e-4ced-8bac-3a149b24110b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feafcf1b5a9844fe9738823fddd07e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:05:02.151865 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763125d-913e-4ced-8bac-3a149b24110b service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:05:02.152148 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763125d-913e-4ced-8bac-3a149b24110b service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:05:02.152430 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763125d-913e-4ced-8bac-3a149b24110b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:05:02.162571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9763125d-913e-4ced-8bac-3a149b24110b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:05:02.164272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9763125d-913e-4ced-8bac-3a149b24110b service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:05:02.174150 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1338/2674] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Jul 25 20:05:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 25 20:05:09.400488 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68db0fe-3630-43a3-8b5d-a413fc311025 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:05:09.402479 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e68db0fe-3630-43a3-8b5d-a413fc311025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:05:09.492111 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9763125d-913e-4ced-8bac-3a149b24110b service swift] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:05:09.495788 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9763125d-913e-4ced-8bac-3a149b24110b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:05:09.526381 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:05:09.551214 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68db0fe-3630-43a3-8b5d-a413fc311025 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:05:09.556526 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e604f321-cf64-4523-9931-7fb88e58b939 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0422d6695bf14925bf0f993cb16e64ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:05:09.557259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604f321-cf64-4523-9931-7fb88e58b939 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:05:09.557619 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604f321-cf64-4523-9931-7fb88e58b939 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:05:09.557990 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604f321-cf64-4523-9931-7fb88e58b939 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:05:09.559944 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e604f321-cf64-4523-9931-7fb88e58b939 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:05:09.572851 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e604f321-cf64-4523-9931-7fb88e58b939 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:05:09.574555 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e604f321-cf64-4523-9931-7fb88e58b939 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:05:09.586034 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1337/2675] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:05:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:05:09.604617 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9763125d-913e-4ced-8bac-3a149b24110b service swift] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:05:09.610897 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9102a35-243f-4723-883d-dca7ed771b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:05:09.611840 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9102a35-243f-4723-883d-dca7ed771b2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:05:09.612163 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9102a35-243f-4723-883d-dca7ed771b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:05:09.612518 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9102a35-243f-4723-883d-dca7ed771b2b admin admin] PATH_INFO: `/v3/users/a0fadb91a8b54ece9ece2b5756be399e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:05:09.613755 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9102a35-243f-4723-883d-dca7ed771b2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0fadb91a8b54ece9ece2b5756be399e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:05:09.615724 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9102a35-243f-4723-883d-dca7ed771b2b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:05:09.663189 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f9102a35-243f-4723-883d-dca7ed771b2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0fadb91a8b54ece9ece2b5756be399e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:05:09.670519 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f9102a35-243f-4723-883d-dca7ed771b2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0fadb91a8b54ece9ece2b5756be399e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:05:09.678409 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f9102a35-243f-4723-883d-dca7ed771b2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0fadb91a8b54ece9ece2b5756be399e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:05:09.679049 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f9102a35-243f-4723-883d-dca7ed771b2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0fadb91a8b54ece9ece2b5756be399e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:05:09.692932 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f9102a35-243f-4723-883d-dca7ed771b2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0fadb91a8b54ece9ece2b5756be399e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:05:09.698877 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1339/2676] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:05:09 2022] DELETE /identity/v3/users/a0fadb91a8b54ece9ece2b5756be399e => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:05:09.872231 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e604f321-cf64-4523-9931-7fb88e58b939 service placement] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:05:09.883594 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e604f321-cf64-4523-9931-7fb88e58b939 service placement] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:05:09.943914 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f0fc0b-76af-4745-abf5-ccbac4012cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:05:09.944775 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f0fc0b-76af-4745-abf5-ccbac4012cc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:05:09.945060 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f0fc0b-76af-4745-abf5-ccbac4012cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:05:09.945335 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f0fc0b-76af-4745-abf5-ccbac4012cc3 admin admin] PATH_INFO: `/v3/projects/e56f16131082413896ec23ec60378226` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:05:09.946502 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f0fc0b-76af-4745-abf5-ccbac4012cc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e56f16131082413896ec23ec60378226)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:05:09.948438 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f0fc0b-76af-4745-abf5-ccbac4012cc3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:05:09.996313 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-68f0fc0b-76af-4745-abf5-ccbac4012cc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e56f16131082413896ec23ec60378226', 'request_id': 'req-68f0fc0b-76af-4745-abf5-ccbac4012cc3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:05:09.998162 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:05:10.002623 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-68f0fc0b-76af-4745-abf5-ccbac4012cc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e56f16131082413896ec23ec60378226', 'request_id': 'req-68f0fc0b-76af-4745-abf5-ccbac4012cc3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:05:10.008611 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-68f0fc0b-76af-4745-abf5-ccbac4012cc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e56f16131082413896ec23ec60378226', 'request_id': 'req-68f0fc0b-76af-4745-abf5-ccbac4012cc3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:05:10.020384 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-68f0fc0b-76af-4745-abf5-ccbac4012cc3 admin admin] The token cache is being invalidate because project e56f16131082413896ec23ec60378226 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:05:10.020676 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-68f0fc0b-76af-4745-abf5-ccbac4012cc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:05:10.021398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-68f0fc0b-76af-4745-abf5-ccbac4012cc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:05:10.022924 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1338/2677] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:05:09 2022] DELETE /identity/v3/projects/e56f16131082413896ec23ec60378226 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:05:10.026912 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9102a35-243f-4723-883d-dca7ed771b2b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:05:10.028394 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f9102a35-243f-4723-883d-dca7ed771b2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:05:10.042050 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9102a35-243f-4723-883d-dca7ed771b2b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:05:10.101358 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac894a-3419-40b1-970e-c929dbe18780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:05:10.102226 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac894a-3419-40b1-970e-c929dbe18780 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:05:10.102509 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac894a-3419-40b1-970e-c929dbe18780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:05:10.102783 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac894a-3419-40b1-970e-c929dbe18780 admin admin] PATH_INFO: `/v3/users/922c141108634d7384b53a92728ae852` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:05:10.104080 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac894a-3419-40b1-970e-c929dbe18780 admin admin] RBAC: Authorizing `identity:delete_user(user_id=922c141108634d7384b53a92728ae852)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:05:10.106025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac894a-3419-40b1-970e-c929dbe18780 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:05:10.146353 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:05:10.157096 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6aac894a-3419-40b1-970e-c929dbe18780 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '922c141108634d7384b53a92728ae852'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:05:10.164477 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6aac894a-3419-40b1-970e-c929dbe18780 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '922c141108634d7384b53a92728ae852'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:05:10.171721 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6aac894a-3419-40b1-970e-c929dbe18780 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '922c141108634d7384b53a92728ae852'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:05:10.172499 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6aac894a-3419-40b1-970e-c929dbe18780 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '922c141108634d7384b53a92728ae852'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:05:10.184896 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6aac894a-3419-40b1-970e-c929dbe18780 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '922c141108634d7384b53a92728ae852'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:05:10.192353 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1340/2678] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:05:10 2022] DELETE /identity/v3/users/922c141108634d7384b53a92728ae852 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:05:10.381539 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f0fc0b-76af-4745-abf5-ccbac4012cc3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:05:10.383172 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-68f0fc0b-76af-4745-abf5-ccbac4012cc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:05:10.397354 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f0fc0b-76af-4745-abf5-ccbac4012cc3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:05:10.469031 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d52775-9335-47c8-92d6-e2e16b74b894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:05:10.470183 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d52775-9335-47c8-92d6-e2e16b74b894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:05:10.470305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d52775-9335-47c8-92d6-e2e16b74b894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:05:10.470627 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d52775-9335-47c8-92d6-e2e16b74b894 admin admin] PATH_INFO: `/v3/projects/5849865a174b40b28021a581ea387847` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:05:10.471906 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d52775-9335-47c8-92d6-e2e16b74b894 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5849865a174b40b28021a581ea387847)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:05:10.474048 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d52775-9335-47c8-92d6-e2e16b74b894 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:05:10.524858 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-99d52775-9335-47c8-92d6-e2e16b74b894 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5849865a174b40b28021a581ea387847', 'request_id': 'req-99d52775-9335-47c8-92d6-e2e16b74b894'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:05:10.531012 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-99d52775-9335-47c8-92d6-e2e16b74b894 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5849865a174b40b28021a581ea387847', 'request_id': 'req-99d52775-9335-47c8-92d6-e2e16b74b894'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:05:10.536938 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-99d52775-9335-47c8-92d6-e2e16b74b894 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5849865a174b40b28021a581ea387847', 'request_id': 'req-99d52775-9335-47c8-92d6-e2e16b74b894'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:05:10.550966 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-99d52775-9335-47c8-92d6-e2e16b74b894 admin admin] The token cache is being invalidate because project 5849865a174b40b28021a581ea387847 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:05:10.551250 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-99d52775-9335-47c8-92d6-e2e16b74b894 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:05:10.551984 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-99d52775-9335-47c8-92d6-e2e16b74b894 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:05:10.553768 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1339/2679] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:05:10 2022] DELETE /identity/v3/projects/5849865a174b40b28021a581ea387847 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:05:20.187444 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=16,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:05:20.548122 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=56,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:06:15.381027 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac894a-3419-40b1-970e-c929dbe18780 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:15.382556 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6aac894a-3419-40b1-970e-c929dbe18780 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:15.458560 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac894a-3419-40b1-970e-c929dbe18780 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:15.509013 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff8de2-5fdf-495e-9570-c446b7c9a182 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:15.509663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff8de2-5fdf-495e-9570-c446b7c9a182 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:15.509955 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff8de2-5fdf-495e-9570-c446b7c9a182 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:15.510233 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff8de2-5fdf-495e-9570-c446b7c9a182 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:15.512049 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-63ff8de2-5fdf-495e-9570-c446b7c9a182 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:15.597024 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff8de2-5fdf-495e-9570-c446b7c9a182 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:15.598487 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff8de2-5fdf-495e-9570-c446b7c9a182 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:15.618315 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1341/2680] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:06:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:06:17.527668 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d52775-9335-47c8-92d6-e2e16b74b894 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:17.529286 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-99d52775-9335-47c8-92d6-e2e16b74b894 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:17.601541 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d52775-9335-47c8-92d6-e2e16b74b894 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:17.605143 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4eb337-9908-48b6-b423-5d3f8c2fae3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:17.605790 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4eb337-9908-48b6-b423-5d3f8c2fae3a service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:17.606076 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4eb337-9908-48b6-b423-5d3f8c2fae3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:17.606354 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4eb337-9908-48b6-b423-5d3f8c2fae3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:17.608144 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9f4eb337-9908-48b6-b423-5d3f8c2fae3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:17.676711 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4eb337-9908-48b6-b423-5d3f8c2fae3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:17.678046 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4eb337-9908-48b6-b423-5d3f8c2fae3a service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:17.688336 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1340/2681] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:06:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:06:21.231906 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff8de2-5fdf-495e-9570-c446b7c9a182 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:21.233361 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-63ff8de2-5fdf-495e-9570-c446b7c9a182 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:21.249207 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff8de2-5fdf-495e-9570-c446b7c9a182 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:21.254033 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6698bb6d-1636-432b-b686-bf07245e5f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:21.254910 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698bb6d-1636-432b-b686-bf07245e5f21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:21.255202 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698bb6d-1636-432b-b686-bf07245e5f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:21.255485 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698bb6d-1636-432b-b686-bf07245e5f21 admin admin] PATH_INFO: `/v3/users/ac2e584f88cd46f280b7fe04aa1382fd` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:21.256885 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6698bb6d-1636-432b-b686-bf07245e5f21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac2e584f88cd46f280b7fe04aa1382fd)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:21.258823 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6698bb6d-1636-432b-b686-bf07245e5f21 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:21.300100 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:06:21.321059 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6698bb6d-1636-432b-b686-bf07245e5f21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac2e584f88cd46f280b7fe04aa1382fd'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:21.328297 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6698bb6d-1636-432b-b686-bf07245e5f21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac2e584f88cd46f280b7fe04aa1382fd'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:21.336408 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6698bb6d-1636-432b-b686-bf07245e5f21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac2e584f88cd46f280b7fe04aa1382fd'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:21.337195 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6698bb6d-1636-432b-b686-bf07245e5f21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac2e584f88cd46f280b7fe04aa1382fd'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:21.349532 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6698bb6d-1636-432b-b686-bf07245e5f21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac2e584f88cd46f280b7fe04aa1382fd'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:21.355866 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1342/2682] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:06:21 2022] DELETE /identity/v3/users/ac2e584f88cd46f280b7fe04aa1382fd => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:06:21.552657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4eb337-9908-48b6-b423-5d3f8c2fae3a service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:21.563044 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4eb337-9908-48b6-b423-5d3f8c2fae3a service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:21.629748 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f53ec5b-015d-424b-9011-9f97da8243dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:21.630606 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f53ec5b-015d-424b-9011-9f97da8243dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:21.630892 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f53ec5b-015d-424b-9011-9f97da8243dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:21.631189 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f53ec5b-015d-424b-9011-9f97da8243dc admin admin] PATH_INFO: `/v3/projects/5ef063ca1eb846d78ff8901d2a3fc0a3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:21.632396 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f53ec5b-015d-424b-9011-9f97da8243dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ef063ca1eb846d78ff8901d2a3fc0a3)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:21.634295 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f53ec5b-015d-424b-9011-9f97da8243dc admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:21.677014 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:06:21.684315 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0f53ec5b-015d-424b-9011-9f97da8243dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ef063ca1eb846d78ff8901d2a3fc0a3', 'request_id': 'req-0f53ec5b-015d-424b-9011-9f97da8243dc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:21.690708 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0f53ec5b-015d-424b-9011-9f97da8243dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ef063ca1eb846d78ff8901d2a3fc0a3', 'request_id': 'req-0f53ec5b-015d-424b-9011-9f97da8243dc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:21.696267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0f53ec5b-015d-424b-9011-9f97da8243dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ef063ca1eb846d78ff8901d2a3fc0a3', 'request_id': 'req-0f53ec5b-015d-424b-9011-9f97da8243dc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:21.709135 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0f53ec5b-015d-424b-9011-9f97da8243dc admin admin] The token cache is being invalidate because project 5ef063ca1eb846d78ff8901d2a3fc0a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:06:21.709422 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0f53ec5b-015d-424b-9011-9f97da8243dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:21.710045 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0f53ec5b-015d-424b-9011-9f97da8243dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:21.711652 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1341/2683] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:06:21 2022] DELETE /identity/v3/projects/5ef063ca1eb846d78ff8901d2a3fc0a3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:06:21.717290 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d52b4f5-763c-499e-9739-d90ca3695a51 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:21.717589 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d52b4f5-763c-499e-9739-d90ca3695a51 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:21.717866 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d52b4f5-763c-499e-9739-d90ca3695a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:21.738335 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3d52b4f5-763c-499e-9739-d90ca3695a51 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:06:21.739561 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3d52b4f5-763c-499e-9739-d90ca3695a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:21.802214 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1343/2684] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:06:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:06:21.809924 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275976c-b849-424e-9816-5fbe4ecabd6f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:21.810213 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275976c-b849-424e-9816-5fbe4ecabd6f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:21.810495 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275976c-b849-424e-9816-5fbe4ecabd6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:21.831106 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9275976c-b849-424e-9816-5fbe4ecabd6f None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:06:21.832471 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9275976c-b849-424e-9816-5fbe4ecabd6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:21.840682 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1342/2685] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:06:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 20:06:21.844897 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d52b4f5-763c-499e-9739-d90ca3695a51 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:21.855774 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d52b4f5-763c-499e-9739-d90ca3695a51 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:21.860587 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c6d020-7fc6-401b-a21f-c06ada4b3a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:21.861503 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c6d020-7fc6-401b-a21f-c06ada4b3a31 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:21.861784 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c6d020-7fc6-401b-a21f-c06ada4b3a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:21.862058 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c6d020-7fc6-401b-a21f-c06ada4b3a31 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:21.862824 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c6d020-7fc6-401b-a21f-c06ada4b3a31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 20:06:21.863129 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c6d020-7fc6-401b-a21f-c06ada4b3a31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:21.864822 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c6d020-7fc6-401b-a21f-c06ada4b3a31 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:21.877823 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1344/2686] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 20:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 20:06:21.882117 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275976c-b849-424e-9816-5fbe4ecabd6f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:21.891400 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275976c-b849-424e-9816-5fbe4ecabd6f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:21.896802 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc719050-0bb8-42fb-83ab-527963b4ca01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:21.897475 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc719050-0bb8-42fb-83ab-527963b4ca01 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:21.897762 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc719050-0bb8-42fb-83ab-527963b4ca01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:21.898043 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc719050-0bb8-42fb-83ab-527963b4ca01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:21.898744 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc719050-0bb8-42fb-83ab-527963b4ca01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:21.900674 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc719050-0bb8-42fb-83ab-527963b4ca01 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:21.930300 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1343/2687] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:06:21 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:06:21.934451 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c6d020-7fc6-401b-a21f-c06ada4b3a31 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:21.944924 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c6d020-7fc6-401b-a21f-c06ada4b3a31 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:22.003905 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84eb09d-6061-4102-ad6e-b464df36741c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:22.004626 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84eb09d-6061-4102-ad6e-b464df36741c admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:22.004918 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84eb09d-6061-4102-ad6e-b464df36741c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:22.005196 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84eb09d-6061-4102-ad6e-b464df36741c admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:22.005910 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84eb09d-6061-4102-ad6e-b464df36741c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:22.007913 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84eb09d-6061-4102-ad6e-b464df36741c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:22.035218 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1345/2688] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:06:21 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:06:22.036506 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc719050-0bb8-42fb-83ab-527963b4ca01 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:22.047533 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc719050-0bb8-42fb-83ab-527963b4ca01 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:22.052931 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d469afc2-6213-487c-975a-c29dac97ea9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:22.053660 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469afc2-6213-487c-975a-c29dac97ea9c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:22.053953 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469afc2-6213-487c-975a-c29dac97ea9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:22.054214 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469afc2-6213-487c-975a-c29dac97ea9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:22.055103 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d469afc2-6213-487c-975a-c29dac97ea9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:22.056745 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d469afc2-6213-487c-975a-c29dac97ea9c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:22.067474 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1344/2689] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:06:22.071474 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84eb09d-6061-4102-ad6e-b464df36741c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:22.083671 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84eb09d-6061-4102-ad6e-b464df36741c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:22.091232 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46625a8e-1c15-4756-b429-d37a037f3721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:22.091871 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46625a8e-1c15-4756-b429-d37a037f3721 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:22.092157 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46625a8e-1c15-4756-b429-d37a037f3721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:22.092437 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46625a8e-1c15-4756-b429-d37a037f3721 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:22.093181 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46625a8e-1c15-4756-b429-d37a037f3721 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:22.095369 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46625a8e-1c15-4756-b429-d37a037f3721 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:22.107978 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1346/2690] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:06:22.113008 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d469afc2-6213-487c-975a-c29dac97ea9c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:22.124349 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d469afc2-6213-487c-975a-c29dac97ea9c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:22.129551 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da246137-18c5-484e-ae6b-85ed0caa6498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:22.130430 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da246137-18c5-484e-ae6b-85ed0caa6498 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:22.130708 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da246137-18c5-484e-ae6b-85ed0caa6498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:22.130983 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da246137-18c5-484e-ae6b-85ed0caa6498 admin admin] PATH_INFO: `/v3/projects/c1c07d6187624229846014d22f2b60fc/users/c3a2b337c7b34d6b9239c80f38d333c6/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:22.146617 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da246137-18c5-484e-ae6b-85ed0caa6498 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1c07d6187624229846014d22f2b60fc, user_id=c3a2b337c7b34d6b9239c80f38d333c6, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:22.149393 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da246137-18c5-484e-ae6b-85ed0caa6498 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:22.165163 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1345/2691] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:06:22 2022] PUT /identity/v3/projects/c1c07d6187624229846014d22f2b60fc/users/c3a2b337c7b34d6b9239c80f38d333c6/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:06:22.175411 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e582a-33a7-45e0-8d8a-8d1746704f6e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:22.175835 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e582a-33a7-45e0-8d8a-8d1746704f6e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:22.176031 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e582a-33a7-45e0-8d8a-8d1746704f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:22.197863 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:06:22.207484 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b58e582a-33a7-45e0-8d8a-8d1746704f6e None None] MFA Rules not processed for user `c3a2b337c7b34d6b9239c80f38d333c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:06:22.209258 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b58e582a-33a7-45e0-8d8a-8d1746704f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:22.276265 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1347/2692] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:06:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:06:22.280881 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da246137-18c5-484e-ae6b-85ed0caa6498 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:22.342635 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da246137-18c5-484e-ae6b-85ed0caa6498 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:22.347599 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224d1fd-892c-4de9-aeff-89b78b9ddcdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:22.348265 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224d1fd-892c-4de9-aeff-89b78b9ddcdc admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:22.348554 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224d1fd-892c-4de9-aeff-89b78b9ddcdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:22.348833 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224d1fd-892c-4de9-aeff-89b78b9ddcdc admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:22.349521 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224d1fd-892c-4de9-aeff-89b78b9ddcdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:22.351413 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224d1fd-892c-4de9-aeff-89b78b9ddcdc admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:22.379179 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1346/2693] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:06:22 2022] POST /identity/v3/projects => generated 371 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:06:22.383276 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e582a-33a7-45e0-8d8a-8d1746704f6e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:22.394163 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e582a-33a7-45e0-8d8a-8d1746704f6e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:22.451996 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9ff1f-3af5-4ce8-8dff-eb3d8f616fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:22.452677 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9ff1f-3af5-4ce8-8dff-eb3d8f616fac admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:22.452970 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9ff1f-3af5-4ce8-8dff-eb3d8f616fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:22.453249 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9ff1f-3af5-4ce8-8dff-eb3d8f616fac admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:22.453945 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a9ff1f-3af5-4ce8-8dff-eb3d8f616fac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:22.456009 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a9ff1f-3af5-4ce8-8dff-eb3d8f616fac admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:22.476705 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1348/2694] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:06:22 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:06:22.480821 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224d1fd-892c-4de9-aeff-89b78b9ddcdc admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:22.491834 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224d1fd-892c-4de9-aeff-89b78b9ddcdc admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:22.496845 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e86d1-876d-4b30-a2a0-7def10d286a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:22.497516 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e86d1-876d-4b30-a2a0-7def10d286a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:22.497830 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e86d1-876d-4b30-a2a0-7def10d286a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:22.498123 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e86d1-876d-4b30-a2a0-7def10d286a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:22.498840 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e86d1-876d-4b30-a2a0-7def10d286a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:22.500534 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e86d1-876d-4b30-a2a0-7def10d286a1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:22.512162 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1347/2695] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:06:22.516529 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9ff1f-3af5-4ce8-8dff-eb3d8f616fac admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:22.527741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9ff1f-3af5-4ce8-8dff-eb3d8f616fac admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:22.533201 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd029426-426e-40e6-af01-011788cb9945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:22.534119 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd029426-426e-40e6-af01-011788cb9945 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:22.534403 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd029426-426e-40e6-af01-011788cb9945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:22.534682 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd029426-426e-40e6-af01-011788cb9945 admin admin] PATH_INFO: `/v3/projects/077079aaa8e842c59e9a4fb3e6cfcfa6/users/37668635b0ee44a6ade35375fa44c7a9/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:22.549910 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd029426-426e-40e6-af01-011788cb9945 admin admin] RBAC: Authorizing `identity:create_grant(project_id=077079aaa8e842c59e9a4fb3e6cfcfa6, user_id=37668635b0ee44a6ade35375fa44c7a9, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:22.552983 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd029426-426e-40e6-af01-011788cb9945 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:22.566092 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1349/2696] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:06:22 2022] PUT /identity/v3/projects/077079aaa8e842c59e9a4fb3e6cfcfa6/users/37668635b0ee44a6ade35375fa44c7a9/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:06:22.570121 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e86d1-876d-4b30-a2a0-7def10d286a1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:22.580568 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e86d1-876d-4b30-a2a0-7def10d286a1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:22.622561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:06:22.643954 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086a7569-efb1-4a58-bd1a-6786078b9ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:22.644627 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a7569-efb1-4a58-bd1a-6786078b9ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:22.644916 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a7569-efb1-4a58-bd1a-6786078b9ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:22.645198 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a7569-efb1-4a58-bd1a-6786078b9ba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:22.645923 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086a7569-efb1-4a58-bd1a-6786078b9ba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:22.647535 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086a7569-efb1-4a58-bd1a-6786078b9ba4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:22.658264 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1348/2697] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:06:22.662461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd029426-426e-40e6-af01-011788cb9945 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:22.673333 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd029426-426e-40e6-af01-011788cb9945 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:22.678135 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65e3e1e-af03-429c-86ff-7fde83bc6ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:22.679007 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65e3e1e-af03-429c-86ff-7fde83bc6ef2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:22.679299 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65e3e1e-af03-429c-86ff-7fde83bc6ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:22.679611 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65e3e1e-af03-429c-86ff-7fde83bc6ef2 admin admin] PATH_INFO: `/v3/projects/077079aaa8e842c59e9a4fb3e6cfcfa6/users/37668635b0ee44a6ade35375fa44c7a9/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:22.681613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65e3e1e-af03-429c-86ff-7fde83bc6ef2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=077079aaa8e842c59e9a4fb3e6cfcfa6, user_id=37668635b0ee44a6ade35375fa44c7a9, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:22.684314 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65e3e1e-af03-429c-86ff-7fde83bc6ef2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:22.695450 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1350/2698] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:06:22 2022] PUT /identity/v3/projects/077079aaa8e842c59e9a4fb3e6cfcfa6/users/37668635b0ee44a6ade35375fa44c7a9/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:06:22.707268 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab4e89-58ab-4b54-b2f4-33a4904a9045 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:22.707635 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab4e89-58ab-4b54-b2f4-33a4904a9045 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:22.707883 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab4e89-58ab-4b54-b2f4-33a4904a9045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:22.737358 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d3ab4e89-58ab-4b54-b2f4-33a4904a9045 None None] MFA Rules not processed for user `37668635b0ee44a6ade35375fa44c7a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:06:22.738583 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d3ab4e89-58ab-4b54-b2f4-33a4904a9045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:22.810547 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1349/2699] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:06:22 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:06:22.825358 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65e3e1e-af03-429c-86ff-7fde83bc6ef2 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:22.827113 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d65e3e1e-af03-429c-86ff-7fde83bc6ef2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:22.887788 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65e3e1e-af03-429c-86ff-7fde83bc6ef2 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:22.891482 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8505922c-fa37-4429-ae0c-a26cc1eda552 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:22.892540 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505922c-fa37-4429-ae0c-a26cc1eda552 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:22.892836 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505922c-fa37-4429-ae0c-a26cc1eda552 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:22.893120 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505922c-fa37-4429-ae0c-a26cc1eda552 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:22.900517 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:06:22.902879 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8505922c-fa37-4429-ae0c-a26cc1eda552 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:22.904705 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8505922c-fa37-4429-ae0c-a26cc1eda552 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:22.916512 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1351/2700] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:06:22 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:06:23.316579 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab4e89-58ab-4b54-b2f4-33a4904a9045 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:23.379267 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab4e89-58ab-4b54-b2f4-33a4904a9045 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:23.384957 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6516a715-1a64-4b4e-a678-c517775b559f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:23.385950 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6516a715-1a64-4b4e-a678-c517775b559f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:23.386307 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6516a715-1a64-4b4e-a678-c517775b559f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:23.386633 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6516a715-1a64-4b4e-a678-c517775b559f admin admin] PATH_INFO: `/v3/users/c3a2b337c7b34d6b9239c80f38d333c6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:23.388045 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6516a715-1a64-4b4e-a678-c517775b559f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3a2b337c7b34d6b9239c80f38d333c6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:23.389991 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6516a715-1a64-4b4e-a678-c517775b559f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:23.439669 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6516a715-1a64-4b4e-a678-c517775b559f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3a2b337c7b34d6b9239c80f38d333c6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:23.446553 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6516a715-1a64-4b4e-a678-c517775b559f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3a2b337c7b34d6b9239c80f38d333c6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:23.453502 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6516a715-1a64-4b4e-a678-c517775b559f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3a2b337c7b34d6b9239c80f38d333c6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:23.454130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6516a715-1a64-4b4e-a678-c517775b559f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3a2b337c7b34d6b9239c80f38d333c6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:23.454872 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6516a715-1a64-4b4e-a678-c517775b559f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3a2b337c7b34d6b9239c80f38d333c6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:23.456143 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:06:23.481392 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1350/2701] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:06:23 2022] DELETE /identity/v3/users/c3a2b337c7b34d6b9239c80f38d333c6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:06:23.486403 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b5179-c023-4270-b2ab-f90e24583d33 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:23.486851 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b5179-c023-4270-b2ab-f90e24583d33 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:23.486952 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b5179-c023-4270-b2ab-f90e24583d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:23.507326 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-834b5179-c023-4270-b2ab-f90e24583d33 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:06:23.508569 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-834b5179-c023-4270-b2ab-f90e24583d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:23.567476 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1352/2702] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:06:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:06:23.577147 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6516a715-1a64-4b4e-a678-c517775b559f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:23.578587 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6516a715-1a64-4b4e-a678-c517775b559f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:23.638582 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6516a715-1a64-4b4e-a678-c517775b559f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:23.643205 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddcfad7-5888-4e91-b1a1-73742abed98c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:23.643884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddcfad7-5888-4e91-b1a1-73742abed98c service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:23.644164 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddcfad7-5888-4e91-b1a1-73742abed98c service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:23.644436 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddcfad7-5888-4e91-b1a1-73742abed98c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:23.653875 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddcfad7-5888-4e91-b1a1-73742abed98c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:23.655308 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddcfad7-5888-4e91-b1a1-73742abed98c service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:23.665402 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1351/2703] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:06:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:06:23.942403 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b5179-c023-4270-b2ab-f90e24583d33 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:23.943893 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-834b5179-c023-4270-b2ab-f90e24583d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:23.960755 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b5179-c023-4270-b2ab-f90e24583d33 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:23.966629 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ca849-e629-4888-a026-5d388f372230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:23.967593 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ca849-e629-4888-a026-5d388f372230 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:23.967781 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ca849-e629-4888-a026-5d388f372230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:23.968100 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ca849-e629-4888-a026-5d388f372230 admin admin] PATH_INFO: `/v3/projects/c1c07d6187624229846014d22f2b60fc` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:23.969257 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6ca849-e629-4888-a026-5d388f372230 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1c07d6187624229846014d22f2b60fc)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:23.972473 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6ca849-e629-4888-a026-5d388f372230 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:24.022109 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8b6ca849-e629-4888-a026-5d388f372230 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1c07d6187624229846014d22f2b60fc', 'request_id': 'req-8b6ca849-e629-4888-a026-5d388f372230'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:24.029593 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8b6ca849-e629-4888-a026-5d388f372230 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1c07d6187624229846014d22f2b60fc', 'request_id': 'req-8b6ca849-e629-4888-a026-5d388f372230'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:24.035062 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8b6ca849-e629-4888-a026-5d388f372230 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1c07d6187624229846014d22f2b60fc', 'request_id': 'req-8b6ca849-e629-4888-a026-5d388f372230'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:24.046717 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8b6ca849-e629-4888-a026-5d388f372230 admin admin] The token cache is being invalidate because project c1c07d6187624229846014d22f2b60fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:06:24.047006 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8b6ca849-e629-4888-a026-5d388f372230 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:24.047699 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8b6ca849-e629-4888-a026-5d388f372230 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:24.050050 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1353/2704] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:06:23 2022] DELETE /identity/v3/projects/c1c07d6187624229846014d22f2b60fc => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:06:24.054233 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddcfad7-5888-4e91-b1a1-73742abed98c service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:24.065574 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddcfad7-5888-4e91-b1a1-73742abed98c service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:24.130386 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa43bb6-1c55-4d8d-8d4f-377aab619b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:24.131342 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa43bb6-1c55-4d8d-8d4f-377aab619b39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:24.131785 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa43bb6-1c55-4d8d-8d4f-377aab619b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:24.132100 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa43bb6-1c55-4d8d-8d4f-377aab619b39 admin admin] PATH_INFO: `/v3/users/37668635b0ee44a6ade35375fa44c7a9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:24.133526 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa43bb6-1c55-4d8d-8d4f-377aab619b39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37668635b0ee44a6ade35375fa44c7a9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:24.135423 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa43bb6-1c55-4d8d-8d4f-377aab619b39 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:24.193696 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3aa43bb6-1c55-4d8d-8d4f-377aab619b39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37668635b0ee44a6ade35375fa44c7a9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:24.200436 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3aa43bb6-1c55-4d8d-8d4f-377aab619b39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37668635b0ee44a6ade35375fa44c7a9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:24.208726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3aa43bb6-1c55-4d8d-8d4f-377aab619b39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37668635b0ee44a6ade35375fa44c7a9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:24.209451 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3aa43bb6-1c55-4d8d-8d4f-377aab619b39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37668635b0ee44a6ade35375fa44c7a9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:24.210058 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3aa43bb6-1c55-4d8d-8d4f-377aab619b39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37668635b0ee44a6ade35375fa44c7a9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:24.228118 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1352/2705] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:06:24 2022] DELETE /identity/v3/users/37668635b0ee44a6ade35375fa44c7a9 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:06:24.514293 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ca849-e629-4888-a026-5d388f372230 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:24.515756 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8b6ca849-e629-4888-a026-5d388f372230 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:24.528859 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ca849-e629-4888-a026-5d388f372230 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:24.590773 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ccab36-bba8-41e0-a088-b6a0d76025bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:24.591683 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ccab36-bba8-41e0-a088-b6a0d76025bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:24.591992 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ccab36-bba8-41e0-a088-b6a0d76025bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:24.592249 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ccab36-bba8-41e0-a088-b6a0d76025bd admin admin] PATH_INFO: `/v3/projects/077079aaa8e842c59e9a4fb3e6cfcfa6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:24.593448 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ccab36-bba8-41e0-a088-b6a0d76025bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=077079aaa8e842c59e9a4fb3e6cfcfa6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:24.595427 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ccab36-bba8-41e0-a088-b6a0d76025bd admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:24.619489 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:06:24.645146 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-14ccab36-bba8-41e0-a088-b6a0d76025bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '077079aaa8e842c59e9a4fb3e6cfcfa6', 'request_id': 'req-14ccab36-bba8-41e0-a088-b6a0d76025bd'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:24.653529 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-14ccab36-bba8-41e0-a088-b6a0d76025bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '077079aaa8e842c59e9a4fb3e6cfcfa6', 'request_id': 'req-14ccab36-bba8-41e0-a088-b6a0d76025bd'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:24.659890 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-14ccab36-bba8-41e0-a088-b6a0d76025bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '077079aaa8e842c59e9a4fb3e6cfcfa6', 'request_id': 'req-14ccab36-bba8-41e0-a088-b6a0d76025bd'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:24.673998 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-14ccab36-bba8-41e0-a088-b6a0d76025bd admin admin] The token cache is being invalidate because project 077079aaa8e842c59e9a4fb3e6cfcfa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:06:24.674297 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-14ccab36-bba8-41e0-a088-b6a0d76025bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:24.675043 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-14ccab36-bba8-41e0-a088-b6a0d76025bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:24.676740 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1354/2706] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:06:24 2022] DELETE /identity/v3/projects/077079aaa8e842c59e9a4fb3e6cfcfa6 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:06:24.682821 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c300f-4dd2-464d-9f11-3edd073fe27f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:24.683306 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c300f-4dd2-464d-9f11-3edd073fe27f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:24.683386 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c300f-4dd2-464d-9f11-3edd073fe27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:24.692513 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:06:24.709240 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-dd8c300f-4dd2-464d-9f11-3edd073fe27f None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:06:24.711039 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-dd8c300f-4dd2-464d-9f11-3edd073fe27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:24.784283 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1353/2707] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:06:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 20:06:24.792059 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745be8b-a48d-4b7d-a821-6b0a2833f2c8 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:24.792354 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745be8b-a48d-4b7d-a821-6b0a2833f2c8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:24.792645 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745be8b-a48d-4b7d-a821-6b0a2833f2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:24.818364 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d745be8b-a48d-4b7d-a821-6b0a2833f2c8 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:06:24.819637 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d745be8b-a48d-4b7d-a821-6b0a2833f2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:24.828089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1355/2708] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:06:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:06:24.833821 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c300f-4dd2-464d-9f11-3edd073fe27f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:24.847164 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c300f-4dd2-464d-9f11-3edd073fe27f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:24.853421 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827d5cb-8aca-4d9c-873c-c4bee44e2c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:24.854240 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827d5cb-8aca-4d9c-873c-c4bee44e2c57 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:24.854565 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827d5cb-8aca-4d9c-873c-c4bee44e2c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:24.854851 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827d5cb-8aca-4d9c-873c-c4bee44e2c57 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:24.856005 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d827d5cb-8aca-4d9c-873c-c4bee44e2c57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 20:06:24.856501 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d827d5cb-8aca-4d9c-873c-c4bee44e2c57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:24.858592 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d827d5cb-8aca-4d9c-873c-c4bee44e2c57 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:24.874287 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1354/2709] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 20:06:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 20:06:24.880496 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d745be8b-a48d-4b7d-a821-6b0a2833f2c8 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:24.891133 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d745be8b-a48d-4b7d-a821-6b0a2833f2c8 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:24.897650 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac488039-1598-401d-aa22-b424ae00fb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:24.898254 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac488039-1598-401d-aa22-b424ae00fb78 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:24.898558 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac488039-1598-401d-aa22-b424ae00fb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:24.898813 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac488039-1598-401d-aa22-b424ae00fb78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:24.899567 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac488039-1598-401d-aa22-b424ae00fb78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:24.901466 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac488039-1598-401d-aa22-b424ae00fb78 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:24.934740 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1356/2710] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:06:24 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:06:24.940928 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827d5cb-8aca-4d9c-873c-c4bee44e2c57 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:24.951975 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827d5cb-8aca-4d9c-873c-c4bee44e2c57 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:25.021653 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b41e65-f0c5-4eff-bf1a-013004b5d9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:25.022343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b41e65-f0c5-4eff-bf1a-013004b5d9a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:25.022632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b41e65-f0c5-4eff-bf1a-013004b5d9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:25.022911 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b41e65-f0c5-4eff-bf1a-013004b5d9a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:25.023753 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b41e65-f0c5-4eff-bf1a-013004b5d9a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:25.025687 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b41e65-f0c5-4eff-bf1a-013004b5d9a5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:25.050013 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1355/2711] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:06:24 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:06:25.054454 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac488039-1598-401d-aa22-b424ae00fb78 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:25.065778 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac488039-1598-401d-aa22-b424ae00fb78 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:25.071179 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec8bca-4193-42c1-beda-50e754349bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:25.072179 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec8bca-4193-42c1-beda-50e754349bdf admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:25.072468 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec8bca-4193-42c1-beda-50e754349bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:25.072748 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec8bca-4193-42c1-beda-50e754349bdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:25.073544 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec8bca-4193-42c1-beda-50e754349bdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:25.075440 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec8bca-4193-42c1-beda-50e754349bdf admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:25.087276 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1357/2712] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:06:25.090927 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b41e65-f0c5-4eff-bf1a-013004b5d9a5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:25.102833 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b41e65-f0c5-4eff-bf1a-013004b5d9a5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:25.107969 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7517dd-7a23-417b-a261-d7535f55302d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:25.108716 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7517dd-7a23-417b-a261-d7535f55302d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:25.108959 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7517dd-7a23-417b-a261-d7535f55302d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:25.109238 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7517dd-7a23-417b-a261-d7535f55302d admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:25.109997 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7517dd-7a23-417b-a261-d7535f55302d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:25.112327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7517dd-7a23-417b-a261-d7535f55302d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:25.124971 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1356/2713] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:06:25.130628 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec8bca-4193-42c1-beda-50e754349bdf admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:25.142406 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec8bca-4193-42c1-beda-50e754349bdf admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:25.147534 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ee8d3-d75e-4c1f-9042-99e6b1c2eb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:25.148483 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ee8d3-d75e-4c1f-9042-99e6b1c2eb56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:25.148714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ee8d3-d75e-4c1f-9042-99e6b1c2eb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:25.148990 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ee8d3-d75e-4c1f-9042-99e6b1c2eb56 admin admin] PATH_INFO: `/v3/projects/5e2169c3415c43e785df7d7bbdf5b9c4/users/a7d28535fd0f48f994fe89447b4bfcbe/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:25.167238 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596ee8d3-d75e-4c1f-9042-99e6b1c2eb56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e2169c3415c43e785df7d7bbdf5b9c4, user_id=a7d28535fd0f48f994fe89447b4bfcbe, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:25.170029 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596ee8d3-d75e-4c1f-9042-99e6b1c2eb56 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:25.187590 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1358/2714] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:06:25 2022] PUT /identity/v3/projects/5e2169c3415c43e785df7d7bbdf5b9c4/users/a7d28535fd0f48f994fe89447b4bfcbe/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:06:25.200014 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d9343-7d66-4d02-be5b-bced2b41335a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:25.200471 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d9343-7d66-4d02-be5b-bced2b41335a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:25.200961 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d9343-7d66-4d02-be5b-bced2b41335a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:25.234869 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-365d9343-7d66-4d02-be5b-bced2b41335a None None] MFA Rules not processed for user `a7d28535fd0f48f994fe89447b4bfcbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:06:25.236633 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-365d9343-7d66-4d02-be5b-bced2b41335a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:25.310894 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:06:25.316892 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1357/2715] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:06:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:06:25.321936 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ee8d3-d75e-4c1f-9042-99e6b1c2eb56 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:25.389614 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ee8d3-d75e-4c1f-9042-99e6b1c2eb56 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:25.395099 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4efa79-36f4-4ece-a1e0-6a5481e5818d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:25.395796 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4efa79-36f4-4ece-a1e0-6a5481e5818d admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:25.396081 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4efa79-36f4-4ece-a1e0-6a5481e5818d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:25.396484 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4efa79-36f4-4ece-a1e0-6a5481e5818d admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:25.397193 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4efa79-36f4-4ece-a1e0-6a5481e5818d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:25.399267 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4efa79-36f4-4ece-a1e0-6a5481e5818d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:25.432124 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1359/2716] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:06:25 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:06:25.436268 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d9343-7d66-4d02-be5b-bced2b41335a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:25.448900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d9343-7d66-4d02-be5b-bced2b41335a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:25.510162 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4479b6-a909-481d-afe2-efe68549dc37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:25.510870 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4479b6-a909-481d-afe2-efe68549dc37 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:25.511377 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4479b6-a909-481d-afe2-efe68549dc37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:25.511829 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4479b6-a909-481d-afe2-efe68549dc37 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:25.512528 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4479b6-a909-481d-afe2-efe68549dc37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:25.514436 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4479b6-a909-481d-afe2-efe68549dc37 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:25.536283 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1358/2717] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:06:25 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:06:25.540506 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4efa79-36f4-4ece-a1e0-6a5481e5818d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:25.552307 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4efa79-36f4-4ece-a1e0-6a5481e5818d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:25.557124 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6983f-de09-4a5b-b88c-237a4f01b3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:25.557797 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6983f-de09-4a5b-b88c-237a4f01b3aa admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:25.558082 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6983f-de09-4a5b-b88c-237a4f01b3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:25.558458 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6983f-de09-4a5b-b88c-237a4f01b3aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:25.559100 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa6983f-de09-4a5b-b88c-237a4f01b3aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:25.560791 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa6983f-de09-4a5b-b88c-237a4f01b3aa admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:25.572402 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1360/2718] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:06:25.577007 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4479b6-a909-481d-afe2-efe68549dc37 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:25.588123 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4479b6-a909-481d-afe2-efe68549dc37 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:25.592679 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f6751-a60b-4506-805a-f1163c788b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:25.593337 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f6751-a60b-4506-805a-f1163c788b21 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:25.593620 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f6751-a60b-4506-805a-f1163c788b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:25.593895 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f6751-a60b-4506-805a-f1163c788b21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:25.594617 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6f6751-a60b-4506-805a-f1163c788b21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:25.597123 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6f6751-a60b-4506-805a-f1163c788b21 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:25.608060 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1359/2719] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:06:25.613576 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6983f-de09-4a5b-b88c-237a4f01b3aa admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:25.624128 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6983f-de09-4a5b-b88c-237a4f01b3aa admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:25.629004 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d6300-6262-4ae0-9987-bae6eb5dd3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:25.629892 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d6300-6262-4ae0-9987-bae6eb5dd3f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:25.630183 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d6300-6262-4ae0-9987-bae6eb5dd3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:25.630467 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d6300-6262-4ae0-9987-bae6eb5dd3f8 admin admin] PATH_INFO: `/v3/projects/9abddf5a1c3943c8a8744ad774a77e00/users/c07ecb9c21f64a999108b50a298be75c/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:25.649244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d6300-6262-4ae0-9987-bae6eb5dd3f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9abddf5a1c3943c8a8744ad774a77e00, user_id=c07ecb9c21f64a999108b50a298be75c, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:25.652031 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d6300-6262-4ae0-9987-bae6eb5dd3f8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:25.655168 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:06:25.668395 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1361/2720] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:06:25 2022] PUT /identity/v3/projects/9abddf5a1c3943c8a8744ad774a77e00/users/c07ecb9c21f64a999108b50a298be75c/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:06:25.681015 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7bcc14-daa6-4139-b762-711cedec62a7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:25.681305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7bcc14-daa6-4139-b762-711cedec62a7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:25.681588 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7bcc14-daa6-4139-b762-711cedec62a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:25.712173 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1f7bcc14-daa6-4139-b762-711cedec62a7 None None] MFA Rules not processed for user `c07ecb9c21f64a999108b50a298be75c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:06:25.713388 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1f7bcc14-daa6-4139-b762-711cedec62a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:25.782340 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1360/2721] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:06:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:06:25.787227 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d6300-6262-4ae0-9987-bae6eb5dd3f8 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:25.853018 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d6300-6262-4ae0-9987-bae6eb5dd3f8 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:25.857806 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ae58d-3edb-4964-9a4b-21cf3f264464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:25.858477 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ae58d-3edb-4964-9a4b-21cf3f264464 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:25.858767 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ae58d-3edb-4964-9a4b-21cf3f264464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:25.859048 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ae58d-3edb-4964-9a4b-21cf3f264464 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:25.859776 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47ae58d-3edb-4964-9a4b-21cf3f264464 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:25.862318 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47ae58d-3edb-4964-9a4b-21cf3f264464 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:25.891063 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1362/2722] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:06:25 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:06:25.895442 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7bcc14-daa6-4139-b762-711cedec62a7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:25.907236 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7bcc14-daa6-4139-b762-711cedec62a7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:25.966682 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801b7b6-e79c-4c48-9f36-a30f474faa96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:25.967350 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801b7b6-e79c-4c48-9f36-a30f474faa96 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:25.967669 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801b7b6-e79c-4c48-9f36-a30f474faa96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:25.967952 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801b7b6-e79c-4c48-9f36-a30f474faa96 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:25.968661 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e801b7b6-e79c-4c48-9f36-a30f474faa96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:25.971259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e801b7b6-e79c-4c48-9f36-a30f474faa96 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:25.988863 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:06:26.006111 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1361/2723] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:06:25 2022] POST /identity/v3/users => generated 299 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 20:06:26.010185 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ae58d-3edb-4964-9a4b-21cf3f264464 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:26.029690 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ae58d-3edb-4964-9a4b-21cf3f264464 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:26.035991 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ebe82-bb2d-4a35-a727-f05c63ef8f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:26.036678 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ebe82-bb2d-4a35-a727-f05c63ef8f51 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:26.036966 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ebe82-bb2d-4a35-a727-f05c63ef8f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:26.037244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ebe82-bb2d-4a35-a727-f05c63ef8f51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:26.037984 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475ebe82-bb2d-4a35-a727-f05c63ef8f51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:26.039725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475ebe82-bb2d-4a35-a727-f05c63ef8f51 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:26.051781 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1363/2724] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:06:26.056289 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801b7b6-e79c-4c48-9f36-a30f474faa96 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:26.068029 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801b7b6-e79c-4c48-9f36-a30f474faa96 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:26.073355 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03374f-902c-4f89-8465-f5aded62794c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:26.074258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03374f-902c-4f89-8465-f5aded62794c admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:26.074548 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03374f-902c-4f89-8465-f5aded62794c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:26.074831 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03374f-902c-4f89-8465-f5aded62794c admin admin] PATH_INFO: `/v3/projects/b067b512bbd743ddb0f5a92d68930b29/users/842b71fd7ea04e539aca6bb08e054644/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:26.092346 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03374f-902c-4f89-8465-f5aded62794c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b067b512bbd743ddb0f5a92d68930b29, user_id=842b71fd7ea04e539aca6bb08e054644, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:26.095030 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03374f-902c-4f89-8465-f5aded62794c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:26.114589 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1362/2725] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:06:26 2022] PUT /identity/v3/projects/b067b512bbd743ddb0f5a92d68930b29/users/842b71fd7ea04e539aca6bb08e054644/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:06:26.119526 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ebe82-bb2d-4a35-a727-f05c63ef8f51 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:26.132263 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ebe82-bb2d-4a35-a727-f05c63ef8f51 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:26.192099 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02560280-8656-4152-a7b1-54d1bcffa7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:26.193780 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02560280-8656-4152-a7b1-54d1bcffa7fa admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:26.194071 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02560280-8656-4152-a7b1-54d1bcffa7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:26.194347 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02560280-8656-4152-a7b1-54d1bcffa7fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:26.195077 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02560280-8656-4152-a7b1-54d1bcffa7fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:26.196824 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02560280-8656-4152-a7b1-54d1bcffa7fa admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:26.207448 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1364/2726] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:06:26.211710 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03374f-902c-4f89-8465-f5aded62794c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:26.223452 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03374f-902c-4f89-8465-f5aded62794c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:26.228257 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec36d6-16d9-4b1e-9f47-3beea73110d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:26.229174 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec36d6-16d9-4b1e-9f47-3beea73110d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:26.229460 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec36d6-16d9-4b1e-9f47-3beea73110d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:26.229738 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec36d6-16d9-4b1e-9f47-3beea73110d7 admin admin] PATH_INFO: `/v3/projects/b067b512bbd743ddb0f5a92d68930b29/users/842b71fd7ea04e539aca6bb08e054644/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:26.231774 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ec36d6-16d9-4b1e-9f47-3beea73110d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b067b512bbd743ddb0f5a92d68930b29, user_id=842b71fd7ea04e539aca6bb08e054644, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:26.234561 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ec36d6-16d9-4b1e-9f47-3beea73110d7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:26.247023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1363/2727] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:06:26 2022] PUT /identity/v3/projects/b067b512bbd743ddb0f5a92d68930b29/users/842b71fd7ea04e539aca6bb08e054644/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:06:26.258707 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbcd9af-ab8a-4b16-beb6-6cf70311c22a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:26.259096 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbcd9af-ab8a-4b16-beb6-6cf70311c22a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:26.259385 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbcd9af-ab8a-4b16-beb6-6cf70311c22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:26.292175 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2dbcd9af-ab8a-4b16-beb6-6cf70311c22a None None] MFA Rules not processed for user `842b71fd7ea04e539aca6bb08e054644`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:06:26.293523 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2dbcd9af-ab8a-4b16-beb6-6cf70311c22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:26.348213 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:06:26.368160 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1365/2728] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:06:26 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 20:06:26.378859 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec36d6-16d9-4b1e-9f47-3beea73110d7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:26.380624 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a4ec36d6-16d9-4b1e-9f47-3beea73110d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:26.439190 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec36d6-16d9-4b1e-9f47-3beea73110d7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:26.442840 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f023f-44ae-4f32-bed3-5d2b11bf8071 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:26.443491 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f023f-44ae-4f32-bed3-5d2b11bf8071 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:26.443806 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f023f-44ae-4f32-bed3-5d2b11bf8071 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:26.444087 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f023f-44ae-4f32-bed3-5d2b11bf8071 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:26.453542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5f023f-44ae-4f32-bed3-5d2b11bf8071 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:26.455287 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5f023f-44ae-4f32-bed3-5d2b11bf8071 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:26.464968 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1364/2729] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:06:26 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:06:26.987456 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbcd9af-ab8a-4b16-beb6-6cf70311c22a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:26.989292 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2dbcd9af-ab8a-4b16-beb6-6cf70311c22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:27.003232 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbcd9af-ab8a-4b16-beb6-6cf70311c22a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:27.006948 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea99d30-fa8e-4688-aefb-9782d344fdc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:27.007633 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea99d30-fa8e-4688-aefb-9782d344fdc4 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:27.007947 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea99d30-fa8e-4688-aefb-9782d344fdc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:27.008228 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea99d30-fa8e-4688-aefb-9782d344fdc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:27.066690 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea99d30-fa8e-4688-aefb-9782d344fdc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:27.068223 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea99d30-fa8e-4688-aefb-9782d344fdc4 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:27.077722 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1366/2730] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:06:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:06:31.175808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f023f-44ae-4f32-bed3-5d2b11bf8071 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:31.185073 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f023f-44ae-4f32-bed3-5d2b11bf8071 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:31.188729 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7510ae7-bbd5-4637-ad46-22ecca17ee8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:31.194812 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7510ae7-bbd5-4637-ad46-22ecca17ee8d service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:31.194812 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7510ae7-bbd5-4637-ad46-22ecca17ee8d service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:31.194812 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7510ae7-bbd5-4637-ad46-22ecca17ee8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:31.249845 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7510ae7-bbd5-4637-ad46-22ecca17ee8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:31.251313 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7510ae7-bbd5-4637-ad46-22ecca17ee8d service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:31.261340 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1365/2731] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:06:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:06:33.431283 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea99d30-fa8e-4688-aefb-9782d344fdc4 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:33.507647 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea99d30-fa8e-4688-aefb-9782d344fdc4 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:33.512334 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e3a1b-81a9-4bfb-b323-f6a13fe0bfdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:33.513241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e3a1b-81a9-4bfb-b323-f6a13fe0bfdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:33.513538 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e3a1b-81a9-4bfb-b323-f6a13fe0bfdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:33.513821 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e3a1b-81a9-4bfb-b323-f6a13fe0bfdc admin admin] PATH_INFO: `/v3/users/a7d28535fd0f48f994fe89447b4bfcbe` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:33.515033 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e3a1b-81a9-4bfb-b323-f6a13fe0bfdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7d28535fd0f48f994fe89447b4bfcbe)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:33.517030 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e3a1b-81a9-4bfb-b323-f6a13fe0bfdc admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:33.566425 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8b9e3a1b-81a9-4bfb-b323-f6a13fe0bfdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7d28535fd0f48f994fe89447b4bfcbe'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:33.574788 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8b9e3a1b-81a9-4bfb-b323-f6a13fe0bfdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7d28535fd0f48f994fe89447b4bfcbe'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:33.583879 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8b9e3a1b-81a9-4bfb-b323-f6a13fe0bfdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7d28535fd0f48f994fe89447b4bfcbe'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:33.584590 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8b9e3a1b-81a9-4bfb-b323-f6a13fe0bfdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7d28535fd0f48f994fe89447b4bfcbe'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:33.596510 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8b9e3a1b-81a9-4bfb-b323-f6a13fe0bfdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7d28535fd0f48f994fe89447b4bfcbe'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:33.604899 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1367/2732] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:06:33 2022] DELETE /identity/v3/users/a7d28535fd0f48f994fe89447b4bfcbe => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:06:33.611051 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ad753-26e6-4ba0-aae0-73ea2d707af1 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:33.611552 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ad753-26e6-4ba0-aae0-73ea2d707af1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:33.611661 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ad753-26e6-4ba0-aae0-73ea2d707af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:33.631492 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9f4ad753-26e6-4ba0-aae0-73ea2d707af1 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:06:33.632815 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9f4ad753-26e6-4ba0-aae0-73ea2d707af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:33.693090 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:06:33.703659 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1366/2733] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:06:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:06:33.716169 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e3a1b-81a9-4bfb-b323-f6a13fe0bfdc admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:33.717651 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8b9e3a1b-81a9-4bfb-b323-f6a13fe0bfdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:33.732643 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:06:33.795971 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e3a1b-81a9-4bfb-b323-f6a13fe0bfdc admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:33.800684 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40988fe-b962-45b1-af57-4024d3d0e80f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:33.801475 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40988fe-b962-45b1-af57-4024d3d0e80f service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:33.801766 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40988fe-b962-45b1-af57-4024d3d0e80f service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:33.802075 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40988fe-b962-45b1-af57-4024d3d0e80f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:33.812312 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40988fe-b962-45b1-af57-4024d3d0e80f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:33.813784 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40988fe-b962-45b1-af57-4024d3d0e80f service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:33.825398 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1368/2734] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:06:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:06:34.122679 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4ad753-26e6-4ba0-aae0-73ea2d707af1 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:34.124309 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9f4ad753-26e6-4ba0-aae0-73ea2d707af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:34.138647 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4ad753-26e6-4ba0-aae0-73ea2d707af1 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:34.143426 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08feda2f-98d1-4e53-9e28-c06beceffdaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:34.145195 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08feda2f-98d1-4e53-9e28-c06beceffdaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:34.145630 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08feda2f-98d1-4e53-9e28-c06beceffdaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:34.145920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08feda2f-98d1-4e53-9e28-c06beceffdaa admin admin] PATH_INFO: `/v3/projects/5e2169c3415c43e785df7d7bbdf5b9c4` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:34.147166 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08feda2f-98d1-4e53-9e28-c06beceffdaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e2169c3415c43e785df7d7bbdf5b9c4)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:34.149140 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08feda2f-98d1-4e53-9e28-c06beceffdaa admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:34.202806 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-08feda2f-98d1-4e53-9e28-c06beceffdaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e2169c3415c43e785df7d7bbdf5b9c4', 'request_id': 'req-08feda2f-98d1-4e53-9e28-c06beceffdaa'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:34.209433 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-08feda2f-98d1-4e53-9e28-c06beceffdaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e2169c3415c43e785df7d7bbdf5b9c4', 'request_id': 'req-08feda2f-98d1-4e53-9e28-c06beceffdaa'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:34.215440 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-08feda2f-98d1-4e53-9e28-c06beceffdaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e2169c3415c43e785df7d7bbdf5b9c4', 'request_id': 'req-08feda2f-98d1-4e53-9e28-c06beceffdaa'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:34.228983 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-08feda2f-98d1-4e53-9e28-c06beceffdaa admin admin] The token cache is being invalidate because project 5e2169c3415c43e785df7d7bbdf5b9c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:06:34.229269 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-08feda2f-98d1-4e53-9e28-c06beceffdaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:34.229900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-08feda2f-98d1-4e53-9e28-c06beceffdaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:34.231434 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1367/2735] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:06:34 2022] DELETE /identity/v3/projects/5e2169c3415c43e785df7d7bbdf5b9c4 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:06:34.235615 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40988fe-b962-45b1-af57-4024d3d0e80f service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:34.247010 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40988fe-b962-45b1-af57-4024d3d0e80f service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:34.310065 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9972c-b529-4bd1-822d-7413e0d549de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:34.310945 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9972c-b529-4bd1-822d-7413e0d549de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:34.311233 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9972c-b529-4bd1-822d-7413e0d549de admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:34.311541 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9972c-b529-4bd1-822d-7413e0d549de admin admin] PATH_INFO: `/v3/users/c07ecb9c21f64a999108b50a298be75c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:34.312749 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e9972c-b529-4bd1-822d-7413e0d549de admin admin] RBAC: Authorizing `identity:delete_user(user_id=c07ecb9c21f64a999108b50a298be75c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:34.314691 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e9972c-b529-4bd1-822d-7413e0d549de admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:34.363218 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f0e9972c-b529-4bd1-822d-7413e0d549de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c07ecb9c21f64a999108b50a298be75c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:34.371565 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f0e9972c-b529-4bd1-822d-7413e0d549de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c07ecb9c21f64a999108b50a298be75c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:34.379150 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f0e9972c-b529-4bd1-822d-7413e0d549de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c07ecb9c21f64a999108b50a298be75c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:34.380085 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f0e9972c-b529-4bd1-822d-7413e0d549de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c07ecb9c21f64a999108b50a298be75c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:34.392945 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f0e9972c-b529-4bd1-822d-7413e0d549de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c07ecb9c21f64a999108b50a298be75c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:34.399690 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1369/2736] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:06:34 2022] DELETE /identity/v3/users/c07ecb9c21f64a999108b50a298be75c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:06:34.707809 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08feda2f-98d1-4e53-9e28-c06beceffdaa admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:34.709378 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-08feda2f-98d1-4e53-9e28-c06beceffdaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:34.726168 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08feda2f-98d1-4e53-9e28-c06beceffdaa admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:34.800008 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e7c17-f5ad-48d0-81f1-b845d4b520fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:34.800962 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e7c17-f5ad-48d0-81f1-b845d4b520fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:34.801249 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e7c17-f5ad-48d0-81f1-b845d4b520fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:34.801663 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e7c17-f5ad-48d0-81f1-b845d4b520fa admin admin] PATH_INFO: `/v3/projects/9abddf5a1c3943c8a8744ad774a77e00` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:34.802877 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285e7c17-f5ad-48d0-81f1-b845d4b520fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=9abddf5a1c3943c8a8744ad774a77e00)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:34.805044 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285e7c17-f5ad-48d0-81f1-b845d4b520fa admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:34.854008 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-285e7c17-f5ad-48d0-81f1-b845d4b520fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9abddf5a1c3943c8a8744ad774a77e00', 'request_id': 'req-285e7c17-f5ad-48d0-81f1-b845d4b520fa'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:34.860356 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-285e7c17-f5ad-48d0-81f1-b845d4b520fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9abddf5a1c3943c8a8744ad774a77e00', 'request_id': 'req-285e7c17-f5ad-48d0-81f1-b845d4b520fa'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:34.865955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-285e7c17-f5ad-48d0-81f1-b845d4b520fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9abddf5a1c3943c8a8744ad774a77e00', 'request_id': 'req-285e7c17-f5ad-48d0-81f1-b845d4b520fa'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:34.879661 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-285e7c17-f5ad-48d0-81f1-b845d4b520fa admin admin] The token cache is being invalidate because project 9abddf5a1c3943c8a8744ad774a77e00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:06:34.880013 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-285e7c17-f5ad-48d0-81f1-b845d4b520fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:34.880644 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-285e7c17-f5ad-48d0-81f1-b845d4b520fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:34.882199 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1368/2737] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:06:34 2022] DELETE /identity/v3/projects/9abddf5a1c3943c8a8744ad774a77e00 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:06:34.886202 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9972c-b529-4bd1-822d-7413e0d549de admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:34.887714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f0e9972c-b529-4bd1-822d-7413e0d549de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:34.901619 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9972c-b529-4bd1-822d-7413e0d549de admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:34.932574 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:06:34.965448 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40a031-ff42-4f64-930e-2fcebeb4743d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:34.966464 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40a031-ff42-4f64-930e-2fcebeb4743d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:34.966756 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40a031-ff42-4f64-930e-2fcebeb4743d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:34.967038 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40a031-ff42-4f64-930e-2fcebeb4743d admin admin] PATH_INFO: `/v3/users/842b71fd7ea04e539aca6bb08e054644` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:34.968291 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad40a031-ff42-4f64-930e-2fcebeb4743d admin admin] RBAC: Authorizing `identity:delete_user(user_id=842b71fd7ea04e539aca6bb08e054644)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:34.970240 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad40a031-ff42-4f64-930e-2fcebeb4743d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:35.019112 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ad40a031-ff42-4f64-930e-2fcebeb4743d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '842b71fd7ea04e539aca6bb08e054644'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:35.027099 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ad40a031-ff42-4f64-930e-2fcebeb4743d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '842b71fd7ea04e539aca6bb08e054644'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:35.034989 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ad40a031-ff42-4f64-930e-2fcebeb4743d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '842b71fd7ea04e539aca6bb08e054644'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:35.035694 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ad40a031-ff42-4f64-930e-2fcebeb4743d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '842b71fd7ea04e539aca6bb08e054644'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:35.047663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ad40a031-ff42-4f64-930e-2fcebeb4743d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '842b71fd7ea04e539aca6bb08e054644'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:35.053582 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1370/2738] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:06:34 2022] DELETE /identity/v3/users/842b71fd7ea04e539aca6bb08e054644 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:06:35.342814 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e7c17-f5ad-48d0-81f1-b845d4b520fa admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:35.344304 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-285e7c17-f5ad-48d0-81f1-b845d4b520fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:35.358091 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e7c17-f5ad-48d0-81f1-b845d4b520fa admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:35.369010 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:06:35.419144 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f785ca-d93f-4a8a-b064-2277c7f538b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:35.420027 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f785ca-d93f-4a8a-b064-2277c7f538b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:35.420315 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f785ca-d93f-4a8a-b064-2277c7f538b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:35.420596 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f785ca-d93f-4a8a-b064-2277c7f538b5 admin admin] PATH_INFO: `/v3/projects/b067b512bbd743ddb0f5a92d68930b29` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:35.421883 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f785ca-d93f-4a8a-b064-2277c7f538b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b067b512bbd743ddb0f5a92d68930b29)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:35.423721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f785ca-d93f-4a8a-b064-2277c7f538b5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:35.470851 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-86f785ca-d93f-4a8a-b064-2277c7f538b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b067b512bbd743ddb0f5a92d68930b29', 'request_id': 'req-86f785ca-d93f-4a8a-b064-2277c7f538b5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:35.479243 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-86f785ca-d93f-4a8a-b064-2277c7f538b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b067b512bbd743ddb0f5a92d68930b29', 'request_id': 'req-86f785ca-d93f-4a8a-b064-2277c7f538b5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:35.486011 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-86f785ca-d93f-4a8a-b064-2277c7f538b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b067b512bbd743ddb0f5a92d68930b29', 'request_id': 'req-86f785ca-d93f-4a8a-b064-2277c7f538b5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:35.502229 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-86f785ca-d93f-4a8a-b064-2277c7f538b5 admin admin] The token cache is being invalidate because project b067b512bbd743ddb0f5a92d68930b29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:06:35.502520 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-86f785ca-d93f-4a8a-b064-2277c7f538b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:35.503683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-86f785ca-d93f-4a8a-b064-2277c7f538b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:35.505373 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1369/2739] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:06:35 2022] DELETE /identity/v3/projects/b067b512bbd743ddb0f5a92d68930b29 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:06:35.511557 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20edb7e1-faba-4681-8057-878879a2dc90 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:35.511854 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20edb7e1-faba-4681-8057-878879a2dc90 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:35.512138 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20edb7e1-faba-4681-8057-878879a2dc90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:35.533045 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-20edb7e1-faba-4681-8057-878879a2dc90 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:06:35.534828 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-20edb7e1-faba-4681-8057-878879a2dc90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:35.605619 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1371/2740] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:06:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:06:35.616898 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfc6f0-6045-47ff-90bd-0e4edb7b8252 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:35.617208 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfc6f0-6045-47ff-90bd-0e4edb7b8252 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:35.617526 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfc6f0-6045-47ff-90bd-0e4edb7b8252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:35.638984 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-92cfc6f0-6045-47ff-90bd-0e4edb7b8252 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:06:35.640258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-92cfc6f0-6045-47ff-90bd-0e4edb7b8252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:35.649348 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1370/2741] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:06:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:06:35.653935 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20edb7e1-faba-4681-8057-878879a2dc90 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:35.669245 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20edb7e1-faba-4681-8057-878879a2dc90 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:35.674044 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d4893-e794-4234-a1b8-154d995152ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:35.674711 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d4893-e794-4234-a1b8-154d995152ac admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:35.675048 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d4893-e794-4234-a1b8-154d995152ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:35.675541 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3d4893-e794-4234-a1b8-154d995152ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:35.676298 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3d4893-e794-4234-a1b8-154d995152ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 20:06:35.676928 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3d4893-e794-4234-a1b8-154d995152ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:35.678225 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3d4893-e794-4234-a1b8-154d995152ac admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:35.693968 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1372/2742] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 20:06:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 20:06:35.696436 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cfc6f0-6045-47ff-90bd-0e4edb7b8252 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:35.706729 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cfc6f0-6045-47ff-90bd-0e4edb7b8252 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:35.712308 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba40ad7-a81d-45d8-84dc-b337b95943ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:35.713230 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba40ad7-a81d-45d8-84dc-b337b95943ff admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:35.713518 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba40ad7-a81d-45d8-84dc-b337b95943ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:35.713832 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba40ad7-a81d-45d8-84dc-b337b95943ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:35.714587 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba40ad7-a81d-45d8-84dc-b337b95943ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:35.716642 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba40ad7-a81d-45d8-84dc-b337b95943ff admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:35.749067 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1371/2743] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:06:35 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:06:35.754156 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d4893-e794-4234-a1b8-154d995152ac admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:35.764809 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3d4893-e794-4234-a1b8-154d995152ac admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:35.805411 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:06:35.834482 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99866972-06d9-460a-a627-4040a1debe8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:35.835862 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99866972-06d9-460a-a627-4040a1debe8b admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:35.836292 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99866972-06d9-460a-a627-4040a1debe8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:35.836708 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99866972-06d9-460a-a627-4040a1debe8b admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:35.838028 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99866972-06d9-460a-a627-4040a1debe8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:35.846307 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99866972-06d9-460a-a627-4040a1debe8b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:35.879438 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1373/2744] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:06:35 2022] POST /identity/v3/users => generated 283 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:06:35.880093 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba40ad7-a81d-45d8-84dc-b337b95943ff admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:35.897705 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba40ad7-a81d-45d8-84dc-b337b95943ff admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:35.909157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc47aa-15db-4fad-b611-4f61864ca116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:35.909157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc47aa-15db-4fad-b611-4f61864ca116 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:35.909157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc47aa-15db-4fad-b611-4f61864ca116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:35.909157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc47aa-15db-4fad-b611-4f61864ca116 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:35.909795 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc47aa-15db-4fad-b611-4f61864ca116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:35.910845 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc47aa-15db-4fad-b611-4f61864ca116 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:35.922973 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1372/2745] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 20:06:35.927925 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99866972-06d9-460a-a627-4040a1debe8b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:35.938657 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99866972-06d9-460a-a627-4040a1debe8b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:35.945837 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48318330-a7f1-4fb6-91ed-b1f94fffc858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:35.946521 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48318330-a7f1-4fb6-91ed-b1f94fffc858 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:35.946811 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48318330-a7f1-4fb6-91ed-b1f94fffc858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:35.947092 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48318330-a7f1-4fb6-91ed-b1f94fffc858 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:35.947889 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48318330-a7f1-4fb6-91ed-b1f94fffc858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:35.949554 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48318330-a7f1-4fb6-91ed-b1f94fffc858 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:35.962306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1374/2746] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:06:35.966602 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc47aa-15db-4fad-b611-4f61864ca116 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:35.977499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc47aa-15db-4fad-b611-4f61864ca116 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:35.982333 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415ef04-a4d2-4222-9069-399bcbd1a49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:35.983218 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415ef04-a4d2-4222-9069-399bcbd1a49a admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:35.983498 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415ef04-a4d2-4222-9069-399bcbd1a49a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:35.983806 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415ef04-a4d2-4222-9069-399bcbd1a49a admin admin] PATH_INFO: `/v3/projects/8d14aa589236406fa73bba07f0367570/users/dfb531ff3b5543a1bf5564c6b1c6d2b8/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:36.000180 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b415ef04-a4d2-4222-9069-399bcbd1a49a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d14aa589236406fa73bba07f0367570, user_id=dfb531ff3b5543a1bf5564c6b1c6d2b8, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:36.002855 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b415ef04-a4d2-4222-9069-399bcbd1a49a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:36.018388 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1373/2747] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:06:35 2022] PUT /identity/v3/projects/8d14aa589236406fa73bba07f0367570/users/dfb531ff3b5543a1bf5564c6b1c6d2b8/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:06:36.030483 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242ae06-2047-4afa-9100-3ecc9ef56e1c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:36.030780 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242ae06-2047-4afa-9100-3ecc9ef56e1c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:36.031140 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242ae06-2047-4afa-9100-3ecc9ef56e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:36.061056 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6242ae06-2047-4afa-9100-3ecc9ef56e1c None None] MFA Rules not processed for user `dfb531ff3b5543a1bf5564c6b1c6d2b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:06:36.062291 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6242ae06-2047-4afa-9100-3ecc9ef56e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:36.135805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1375/2748] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:06:36 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:06:36.140206 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415ef04-a4d2-4222-9069-399bcbd1a49a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:36.186031 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:06:36.208626 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415ef04-a4d2-4222-9069-399bcbd1a49a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:36.213534 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb41c53-5f6e-4184-903e-772640ee4c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:36.214200 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb41c53-5f6e-4184-903e-772640ee4c37 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:36.214485 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb41c53-5f6e-4184-903e-772640ee4c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:36.214763 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb41c53-5f6e-4184-903e-772640ee4c37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:36.215478 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb41c53-5f6e-4184-903e-772640ee4c37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:36.217407 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb41c53-5f6e-4184-903e-772640ee4c37 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:36.245618 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1374/2749] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:06:36 2022] POST /identity/v3/projects => generated 365 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:06:36.249705 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242ae06-2047-4afa-9100-3ecc9ef56e1c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:36.262094 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242ae06-2047-4afa-9100-3ecc9ef56e1c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:36.325565 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d205ac4a-2e36-4c55-9862-6effa1163070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:36.326263 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205ac4a-2e36-4c55-9862-6effa1163070 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:36.326552 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205ac4a-2e36-4c55-9862-6effa1163070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:36.326954 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205ac4a-2e36-4c55-9862-6effa1163070 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:36.327714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d205ac4a-2e36-4c55-9862-6effa1163070 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:36.329704 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d205ac4a-2e36-4c55-9862-6effa1163070 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:36.352810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1376/2750] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:06:36 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 20:06:36.356958 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb41c53-5f6e-4184-903e-772640ee4c37 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:36.368338 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb41c53-5f6e-4184-903e-772640ee4c37 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:36.373880 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393648c-38e9-4601-ab43-1ae499c13249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:36.374683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393648c-38e9-4601-ab43-1ae499c13249 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:36.375064 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393648c-38e9-4601-ab43-1ae499c13249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:36.375489 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393648c-38e9-4601-ab43-1ae499c13249 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:36.376306 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a393648c-38e9-4601-ab43-1ae499c13249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:36.378793 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a393648c-38e9-4601-ab43-1ae499c13249 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:36.390513 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1375/2751] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:06:36.395562 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d205ac4a-2e36-4c55-9862-6effa1163070 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:36.407635 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d205ac4a-2e36-4c55-9862-6effa1163070 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:36.415161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e58b3-8a35-4057-a0e4-8011f1a8fc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:36.416274 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e58b3-8a35-4057-a0e4-8011f1a8fc13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:36.416565 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e58b3-8a35-4057-a0e4-8011f1a8fc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:36.416905 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e58b3-8a35-4057-a0e4-8011f1a8fc13 admin admin] PATH_INFO: `/v3/projects/b3af3599891b48f5935ef353dabc23ef/users/c13321a646644fcbb2d5e38c6e6d3f53/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:36.435995 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e58b3-8a35-4057-a0e4-8011f1a8fc13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3af3599891b48f5935ef353dabc23ef, user_id=c13321a646644fcbb2d5e38c6e6d3f53, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:36.438640 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e58b3-8a35-4057-a0e4-8011f1a8fc13 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:36.454346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1377/2752] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:06:36 2022] PUT /identity/v3/projects/b3af3599891b48f5935ef353dabc23ef/users/c13321a646644fcbb2d5e38c6e6d3f53/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:06:36.458851 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393648c-38e9-4601-ab43-1ae499c13249 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:36.469315 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393648c-38e9-4601-ab43-1ae499c13249 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:36.540589 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c9d803-5100-44c8-a8b0-cf7277e24ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:36.541275 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c9d803-5100-44c8-a8b0-cf7277e24ecc admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:36.541592 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c9d803-5100-44c8-a8b0-cf7277e24ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:36.541853 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c9d803-5100-44c8-a8b0-cf7277e24ecc admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:36.542730 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c9d803-5100-44c8-a8b0-cf7277e24ecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:36.544488 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c9d803-5100-44c8-a8b0-cf7277e24ecc admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:36.556849 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1376/2753] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:06:36.561109 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e58b3-8a35-4057-a0e4-8011f1a8fc13 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:36.572324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e58b3-8a35-4057-a0e4-8011f1a8fc13 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:36.577025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d215e080-ea45-4a29-bf64-acf795ffdbcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:36.577913 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215e080-ea45-4a29-bf64-acf795ffdbcc admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:36.578248 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215e080-ea45-4a29-bf64-acf795ffdbcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:36.578547 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215e080-ea45-4a29-bf64-acf795ffdbcc admin admin] PATH_INFO: `/v3/projects/b3af3599891b48f5935ef353dabc23ef/users/c13321a646644fcbb2d5e38c6e6d3f53/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:36.580550 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d215e080-ea45-4a29-bf64-acf795ffdbcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3af3599891b48f5935ef353dabc23ef, user_id=c13321a646644fcbb2d5e38c6e6d3f53, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:36.583701 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d215e080-ea45-4a29-bf64-acf795ffdbcc admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:36.589281 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:06:36.595271 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1378/2754] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:06:36 2022] PUT /identity/v3/projects/b3af3599891b48f5935ef353dabc23ef/users/c13321a646644fcbb2d5e38c6e6d3f53/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 20:06:36.606606 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66204f66-e532-467c-bb0b-65afd39b5e18 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:36.606899 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66204f66-e532-467c-bb0b-65afd39b5e18 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:36.607180 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66204f66-e532-467c-bb0b-65afd39b5e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:36.639238 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-66204f66-e532-467c-bb0b-65afd39b5e18 None None] MFA Rules not processed for user `c13321a646644fcbb2d5e38c6e6d3f53`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:06:36.640595 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-66204f66-e532-467c-bb0b-65afd39b5e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:36.709679 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1377/2755] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:06:36 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:06:36.721582 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d215e080-ea45-4a29-bf64-acf795ffdbcc admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:36.724561 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d215e080-ea45-4a29-bf64-acf795ffdbcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:36.793093 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d215e080-ea45-4a29-bf64-acf795ffdbcc admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:36.796801 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9885f3-75f9-4bb0-8935-73e577812f1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:36.797455 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9885f3-75f9-4bb0-8935-73e577812f1a service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:36.797736 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9885f3-75f9-4bb0-8935-73e577812f1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:36.798020 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9885f3-75f9-4bb0-8935-73e577812f1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:36.808199 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9885f3-75f9-4bb0-8935-73e577812f1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:36.809669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9885f3-75f9-4bb0-8935-73e577812f1a service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:36.820567 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1379/2756] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:06:36 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:06:36.910597 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66204f66-e532-467c-bb0b-65afd39b5e18 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:36.922921 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66204f66-e532-467c-bb0b-65afd39b5e18 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:36.927556 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e071d431-485d-4ebb-bb37-8f925198fe4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:36.928222 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071d431-485d-4ebb-bb37-8f925198fe4e service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:36.928513 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071d431-485d-4ebb-bb37-8f925198fe4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:36.928792 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071d431-485d-4ebb-bb37-8f925198fe4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:36.967677 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:06:36.995426 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e071d431-485d-4ebb-bb37-8f925198fe4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:36.997077 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e071d431-485d-4ebb-bb37-8f925198fe4e service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:37.006907 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1378/2757] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:06:36 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:06:41.917113 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9885f3-75f9-4bb0-8935-73e577812f1a service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:41.997412 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9885f3-75f9-4bb0-8935-73e577812f1a service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:42.001960 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf4bce0-e096-4d71-8c9c-c754d74b203a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:42.002957 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf4bce0-e096-4d71-8c9c-c754d74b203a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:42.003788 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf4bce0-e096-4d71-8c9c-c754d74b203a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:42.004072 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf4bce0-e096-4d71-8c9c-c754d74b203a admin admin] PATH_INFO: `/v3/users/dfb531ff3b5543a1bf5564c6b1c6d2b8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:42.005334 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf4bce0-e096-4d71-8c9c-c754d74b203a admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfb531ff3b5543a1bf5564c6b1c6d2b8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:42.007330 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf4bce0-e096-4d71-8c9c-c754d74b203a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:42.064414 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4bf4bce0-e096-4d71-8c9c-c754d74b203a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfb531ff3b5543a1bf5564c6b1c6d2b8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:42.075390 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4bf4bce0-e096-4d71-8c9c-c754d74b203a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfb531ff3b5543a1bf5564c6b1c6d2b8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:42.086944 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4bf4bce0-e096-4d71-8c9c-c754d74b203a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfb531ff3b5543a1bf5564c6b1c6d2b8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:42.087858 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4bf4bce0-e096-4d71-8c9c-c754d74b203a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfb531ff3b5543a1bf5564c6b1c6d2b8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:42.101428 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4bf4bce0-e096-4d71-8c9c-c754d74b203a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfb531ff3b5543a1bf5564c6b1c6d2b8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:42.107823 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1380/2758] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:06:41 2022] DELETE /identity/v3/users/dfb531ff3b5543a1bf5564c6b1c6d2b8 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:06:42.113191 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50662e1-9322-4dc0-ba99-80213d2083c6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:42.113724 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50662e1-9322-4dc0-ba99-80213d2083c6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:42.113840 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50662e1-9322-4dc0-ba99-80213d2083c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:42.135608 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c50662e1-9322-4dc0-ba99-80213d2083c6 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:06:42.136838 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c50662e1-9322-4dc0-ba99-80213d2083c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:42.200032 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1379/2759] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:06:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 20:06:42.212257 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf4bce0-e096-4d71-8c9c-c754d74b203a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:42.213741 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4bf4bce0-e096-4d71-8c9c-c754d74b203a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:42.238949 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:06:42.282501 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf4bce0-e096-4d71-8c9c-c754d74b203a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:42.290072 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c74678-f499-4e90-a41c-a9810786e8ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:42.291172 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c74678-f499-4e90-a41c-a9810786e8ea service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:42.291684 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c74678-f499-4e90-a41c-a9810786e8ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:42.292163 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c74678-f499-4e90-a41c-a9810786e8ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:42.306913 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c74678-f499-4e90-a41c-a9810786e8ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:42.306913 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c74678-f499-4e90-a41c-a9810786e8ea service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:42.317789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1381/2760] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:06:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:06:42.612638 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50662e1-9322-4dc0-ba99-80213d2083c6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:42.614391 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c50662e1-9322-4dc0-ba99-80213d2083c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:42.632294 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50662e1-9322-4dc0-ba99-80213d2083c6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:42.637283 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09539f-0aa1-4e7a-a9f3-1e58c7500dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:42.638927 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09539f-0aa1-4e7a-a9f3-1e58c7500dd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:42.639392 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09539f-0aa1-4e7a-a9f3-1e58c7500dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:42.639718 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09539f-0aa1-4e7a-a9f3-1e58c7500dd9 admin admin] PATH_INFO: `/v3/projects/8d14aa589236406fa73bba07f0367570` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:42.641041 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba09539f-0aa1-4e7a-a9f3-1e58c7500dd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d14aa589236406fa73bba07f0367570)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:42.642957 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba09539f-0aa1-4e7a-a9f3-1e58c7500dd9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:42.697018 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ba09539f-0aa1-4e7a-a9f3-1e58c7500dd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d14aa589236406fa73bba07f0367570', 'request_id': 'req-ba09539f-0aa1-4e7a-a9f3-1e58c7500dd9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:42.704265 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ba09539f-0aa1-4e7a-a9f3-1e58c7500dd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d14aa589236406fa73bba07f0367570', 'request_id': 'req-ba09539f-0aa1-4e7a-a9f3-1e58c7500dd9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:42.711422 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ba09539f-0aa1-4e7a-a9f3-1e58c7500dd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d14aa589236406fa73bba07f0367570', 'request_id': 'req-ba09539f-0aa1-4e7a-a9f3-1e58c7500dd9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:42.727200 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ba09539f-0aa1-4e7a-a9f3-1e58c7500dd9 admin admin] The token cache is being invalidate because project 8d14aa589236406fa73bba07f0367570 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:06:42.727497 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ba09539f-0aa1-4e7a-a9f3-1e58c7500dd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:42.728193 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ba09539f-0aa1-4e7a-a9f3-1e58c7500dd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:42.729804 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1380/2761] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:06:42 2022] DELETE /identity/v3/projects/8d14aa589236406fa73bba07f0367570 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 20:06:42.734733 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c74678-f499-4e90-a41c-a9810786e8ea service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:42.745834 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c74678-f499-4e90-a41c-a9810786e8ea service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:42.816104 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b79b565-b2ad-4f12-a4c2-5ee1f5e4fb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:42.816948 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79b565-b2ad-4f12-a4c2-5ee1f5e4fb5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:42.817239 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79b565-b2ad-4f12-a4c2-5ee1f5e4fb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:42.817526 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79b565-b2ad-4f12-a4c2-5ee1f5e4fb5b admin admin] PATH_INFO: `/v3/users/c13321a646644fcbb2d5e38c6e6d3f53` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:42.818796 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b79b565-b2ad-4f12-a4c2-5ee1f5e4fb5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c13321a646644fcbb2d5e38c6e6d3f53)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:42.820790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b79b565-b2ad-4f12-a4c2-5ee1f5e4fb5b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:42.872824 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3b79b565-b2ad-4f12-a4c2-5ee1f5e4fb5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c13321a646644fcbb2d5e38c6e6d3f53'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:42.881137 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3b79b565-b2ad-4f12-a4c2-5ee1f5e4fb5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c13321a646644fcbb2d5e38c6e6d3f53'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:42.889042 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3b79b565-b2ad-4f12-a4c2-5ee1f5e4fb5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c13321a646644fcbb2d5e38c6e6d3f53'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:42.889768 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3b79b565-b2ad-4f12-a4c2-5ee1f5e4fb5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c13321a646644fcbb2d5e38c6e6d3f53'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:42.902549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3b79b565-b2ad-4f12-a4c2-5ee1f5e4fb5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c13321a646644fcbb2d5e38c6e6d3f53'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:42.910449 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1382/2762] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:06:42 2022] DELETE /identity/v3/users/c13321a646644fcbb2d5e38c6e6d3f53 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:06:43.223673 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09539f-0aa1-4e7a-a9f3-1e58c7500dd9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:43.225398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ba09539f-0aa1-4e7a-a9f3-1e58c7500dd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:43.241769 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09539f-0aa1-4e7a-a9f3-1e58c7500dd9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:43.302962 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:06:43.310635 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1749da8-e1d6-48ff-b210-f2bf653f6ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:43.311876 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1749da8-e1d6-48ff-b210-f2bf653f6ec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:43.312316 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1749da8-e1d6-48ff-b210-f2bf653f6ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:43.312612 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1749da8-e1d6-48ff-b210-f2bf653f6ec8 admin admin] PATH_INFO: `/v3/projects/b3af3599891b48f5935ef353dabc23ef` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:43.314218 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1749da8-e1d6-48ff-b210-f2bf653f6ec8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3af3599891b48f5935ef353dabc23ef)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:43.317351 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1749da8-e1d6-48ff-b210-f2bf653f6ec8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:43.371123 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a1749da8-e1d6-48ff-b210-f2bf653f6ec8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3af3599891b48f5935ef353dabc23ef', 'request_id': 'req-a1749da8-e1d6-48ff-b210-f2bf653f6ec8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:43.378221 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a1749da8-e1d6-48ff-b210-f2bf653f6ec8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3af3599891b48f5935ef353dabc23ef', 'request_id': 'req-a1749da8-e1d6-48ff-b210-f2bf653f6ec8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:43.384209 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a1749da8-e1d6-48ff-b210-f2bf653f6ec8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3af3599891b48f5935ef353dabc23ef', 'request_id': 'req-a1749da8-e1d6-48ff-b210-f2bf653f6ec8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:43.401329 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a1749da8-e1d6-48ff-b210-f2bf653f6ec8 admin admin] The token cache is being invalidate because project b3af3599891b48f5935ef353dabc23ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:06:43.401621 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a1749da8-e1d6-48ff-b210-f2bf653f6ec8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:43.402305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a1749da8-e1d6-48ff-b210-f2bf653f6ec8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:43.404042 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1381/2763] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:06:43 2022] DELETE /identity/v3/projects/b3af3599891b48f5935ef353dabc23ef => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:06:43.410869 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e120d3-f5c9-46fb-ad53-954643bea5c6 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:43.411673 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e120d3-f5c9-46fb-ad53-954643bea5c6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:43.411673 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e120d3-f5c9-46fb-ad53-954643bea5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:43.433303 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f9e120d3-f5c9-46fb-ad53-954643bea5c6 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:06:43.434535 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f9e120d3-f5c9-46fb-ad53-954643bea5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:43.478357 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:06:43.516688 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1383/2764] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:06:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:06:43.525796 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca756a62-52a3-4b9c-8fa2-e76f3c8533ec None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:43.526088 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca756a62-52a3-4b9c-8fa2-e76f3c8533ec None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:43.526376 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca756a62-52a3-4b9c-8fa2-e76f3c8533ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:43.548262 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ca756a62-52a3-4b9c-8fa2-e76f3c8533ec None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:06:43.549535 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ca756a62-52a3-4b9c-8fa2-e76f3c8533ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:43.558138 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1382/2765] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:06:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:06:43.562551 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e120d3-f5c9-46fb-ad53-954643bea5c6 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:43.573601 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e120d3-f5c9-46fb-ad53-954643bea5c6 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:43.578468 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9566c-4ec7-4289-8d38-ba671cff93e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:43.579164 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9566c-4ec7-4289-8d38-ba671cff93e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:43.579453 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9566c-4ec7-4289-8d38-ba671cff93e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:43.579782 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9566c-4ec7-4289-8d38-ba671cff93e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:43.580540 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d9566c-4ec7-4289-8d38-ba671cff93e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 20:06:43.580857 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d9566c-4ec7-4289-8d38-ba671cff93e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:43.583327 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d9566c-4ec7-4289-8d38-ba671cff93e9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:43.597098 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1384/2766] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 20:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 20:06:43.601843 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca756a62-52a3-4b9c-8fa2-e76f3c8533ec None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:43.611623 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca756a62-52a3-4b9c-8fa2-e76f3c8533ec None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:43.617412 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d2762-2f96-4f6b-9485-8367286f9395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:43.618352 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d2762-2f96-4f6b-9485-8367286f9395 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:43.618744 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d2762-2f96-4f6b-9485-8367286f9395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:43.619077 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d2762-2f96-4f6b-9485-8367286f9395 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:43.620005 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135d2762-2f96-4f6b-9485-8367286f9395 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:43.622106 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135d2762-2f96-4f6b-9485-8367286f9395 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:43.656930 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1383/2767] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:06:43 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:06:43.661584 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9566c-4ec7-4289-8d38-ba671cff93e9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:43.672241 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9566c-4ec7-4289-8d38-ba671cff93e9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:43.734839 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b58e56e-d7c2-4188-9b71-7876e51bf32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:43.735544 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b58e56e-d7c2-4188-9b71-7876e51bf32a admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:43.735834 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b58e56e-d7c2-4188-9b71-7876e51bf32a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:43.736113 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b58e56e-d7c2-4188-9b71-7876e51bf32a admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:43.736836 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b58e56e-d7c2-4188-9b71-7876e51bf32a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:43.739558 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b58e56e-d7c2-4188-9b71-7876e51bf32a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:43.766532 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1385/2768] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:06:43 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:06:43.767872 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d2762-2f96-4f6b-9485-8367286f9395 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:43.778919 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d2762-2f96-4f6b-9485-8367286f9395 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:43.783744 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c7299-9170-48d8-8d8c-46748dd2d242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:43.785137 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c7299-9170-48d8-8d8c-46748dd2d242 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:43.785464 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c7299-9170-48d8-8d8c-46748dd2d242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:43.785760 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c7299-9170-48d8-8d8c-46748dd2d242 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:43.786501 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97c7299-9170-48d8-8d8c-46748dd2d242 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:43.788278 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97c7299-9170-48d8-8d8c-46748dd2d242 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:43.801656 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1384/2769] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 20:06:43.806816 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b58e56e-d7c2-4188-9b71-7876e51bf32a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:43.818348 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b58e56e-d7c2-4188-9b71-7876e51bf32a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:43.824400 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea9bac-231c-4074-ad0e-197dab1bab6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:43.825073 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea9bac-231c-4074-ad0e-197dab1bab6b admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:43.825363 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea9bac-231c-4074-ad0e-197dab1bab6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:43.825665 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea9bac-231c-4074-ad0e-197dab1bab6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:43.826381 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea9bac-231c-4074-ad0e-197dab1bab6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:43.828043 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea9bac-231c-4074-ad0e-197dab1bab6b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:43.839669 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1386/2770] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:06:43.844167 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c7299-9170-48d8-8d8c-46748dd2d242 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:43.858433 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c7299-9170-48d8-8d8c-46748dd2d242 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:43.863826 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e148350e-e0b8-48c7-a4d5-bbb8839291f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:43.865032 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e148350e-e0b8-48c7-a4d5-bbb8839291f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:43.865032 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e148350e-e0b8-48c7-a4d5-bbb8839291f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:43.865273 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e148350e-e0b8-48c7-a4d5-bbb8839291f6 admin admin] PATH_INFO: `/v3/projects/5b795671c12b4ac1817149c78575f577/users/17609ecb08c14f20992c7214d64c8c15/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:43.881459 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e148350e-e0b8-48c7-a4d5-bbb8839291f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b795671c12b4ac1817149c78575f577, user_id=17609ecb08c14f20992c7214d64c8c15, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:43.884361 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e148350e-e0b8-48c7-a4d5-bbb8839291f6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:43.898520 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1385/2771] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:06:43 2022] PUT /identity/v3/projects/5b795671c12b4ac1817149c78575f577/users/17609ecb08c14f20992c7214d64c8c15/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:06:43.908884 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046a3e1-fdd5-4cf7-a2b3-98c86cc24d08 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:43.909892 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046a3e1-fdd5-4cf7-a2b3-98c86cc24d08 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:43.910390 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046a3e1-fdd5-4cf7-a2b3-98c86cc24d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:43.940535 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8046a3e1-fdd5-4cf7-a2b3-98c86cc24d08 None None] MFA Rules not processed for user `17609ecb08c14f20992c7214d64c8c15`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:06:43.942090 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8046a3e1-fdd5-4cf7-a2b3-98c86cc24d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:44.142979 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1387/2772] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:06:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:06:44.153080 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e148350e-e0b8-48c7-a4d5-bbb8839291f6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:44.231821 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e148350e-e0b8-48c7-a4d5-bbb8839291f6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:44.236759 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40f870-ece5-496b-bc6e-2b73860ad346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:44.237989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40f870-ece5-496b-bc6e-2b73860ad346 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:44.238271 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40f870-ece5-496b-bc6e-2b73860ad346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:44.238548 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40f870-ece5-496b-bc6e-2b73860ad346 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:44.239276 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b40f870-ece5-496b-bc6e-2b73860ad346 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:44.241295 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b40f870-ece5-496b-bc6e-2b73860ad346 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:44.275323 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1386/2773] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:06:44 2022] POST /identity/v3/projects => generated 371 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:06:44.279594 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8046a3e1-fdd5-4cf7-a2b3-98c86cc24d08 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:44.295000 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8046a3e1-fdd5-4cf7-a2b3-98c86cc24d08 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:44.333081 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:06:44.360488 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fef406-49f7-4485-ab3a-a36e7c8ca635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:44.361175 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fef406-49f7-4485-ab3a-a36e7c8ca635 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:44.361462 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fef406-49f7-4485-ab3a-a36e7c8ca635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:44.361739 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fef406-49f7-4485-ab3a-a36e7c8ca635 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:44.363056 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fef406-49f7-4485-ab3a-a36e7c8ca635 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:44.365561 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fef406-49f7-4485-ab3a-a36e7c8ca635 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:44.389585 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1388/2774] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:06:44 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 20:06:44.393985 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40f870-ece5-496b-bc6e-2b73860ad346 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:44.405661 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40f870-ece5-496b-bc6e-2b73860ad346 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:44.411296 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af96e27-2c39-4ba5-bbcc-ebbee44598fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:44.412053 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af96e27-2c39-4ba5-bbcc-ebbee44598fe admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:44.412346 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af96e27-2c39-4ba5-bbcc-ebbee44598fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:44.412630 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af96e27-2c39-4ba5-bbcc-ebbee44598fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:44.413382 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af96e27-2c39-4ba5-bbcc-ebbee44598fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:44.415086 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af96e27-2c39-4ba5-bbcc-ebbee44598fe admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:44.426160 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:06:44.446674 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1387/2775] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:06:44.448389 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fef406-49f7-4485-ab3a-a36e7c8ca635 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:44.459287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fef406-49f7-4485-ab3a-a36e7c8ca635 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:44.464188 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae525f-4a57-4737-84a3-f3b97d829f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:44.465093 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae525f-4a57-4737-84a3-f3b97d829f36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:44.465384 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae525f-4a57-4737-84a3-f3b97d829f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:44.465666 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae525f-4a57-4737-84a3-f3b97d829f36 admin admin] PATH_INFO: `/v3/projects/2ffe2ead096c48b78b42b0deecc3fb69/users/855c2d0dfe92427dae33dd5615d9106f/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:44.484113 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae525f-4a57-4737-84a3-f3b97d829f36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ffe2ead096c48b78b42b0deecc3fb69, user_id=855c2d0dfe92427dae33dd5615d9106f, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:44.487047 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae525f-4a57-4737-84a3-f3b97d829f36 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:44.503572 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1389/2776] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:06:44 2022] PUT /identity/v3/projects/2ffe2ead096c48b78b42b0deecc3fb69/users/855c2d0dfe92427dae33dd5615d9106f/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 20:06:44.508184 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af96e27-2c39-4ba5-bbcc-ebbee44598fe admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:44.519495 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af96e27-2c39-4ba5-bbcc-ebbee44598fe admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:44.586889 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4e08f-56ea-4d13-aff5-212c0e046a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:44.587772 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4e08f-56ea-4d13-aff5-212c0e046a3c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:44.588064 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4e08f-56ea-4d13-aff5-212c0e046a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:44.588348 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4e08f-56ea-4d13-aff5-212c0e046a3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:44.589090 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d4e08f-56ea-4d13-aff5-212c0e046a3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:44.590827 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d4e08f-56ea-4d13-aff5-212c0e046a3c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:44.602914 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1388/2777] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:06:44.608145 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae525f-4a57-4737-84a3-f3b97d829f36 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:44.628394 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae525f-4a57-4737-84a3-f3b97d829f36 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:44.636895 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341420d4-8b1b-43d4-8a96-bc5dfd4aae91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:44.637803 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341420d4-8b1b-43d4-8a96-bc5dfd4aae91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:44.638088 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341420d4-8b1b-43d4-8a96-bc5dfd4aae91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:44.638507 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341420d4-8b1b-43d4-8a96-bc5dfd4aae91 admin admin] PATH_INFO: `/v3/projects/2ffe2ead096c48b78b42b0deecc3fb69/users/855c2d0dfe92427dae33dd5615d9106f/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:44.640577 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341420d4-8b1b-43d4-8a96-bc5dfd4aae91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ffe2ead096c48b78b42b0deecc3fb69, user_id=855c2d0dfe92427dae33dd5615d9106f, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:44.643277 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341420d4-8b1b-43d4-8a96-bc5dfd4aae91 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:44.655677 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1390/2778] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:06:44 2022] PUT /identity/v3/projects/2ffe2ead096c48b78b42b0deecc3fb69/users/855c2d0dfe92427dae33dd5615d9106f/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:06:44.667660 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad081c4-10b4-464a-aecc-66f31e728e67 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:44.668101 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad081c4-10b4-464a-aecc-66f31e728e67 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:44.668439 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad081c4-10b4-464a-aecc-66f31e728e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:44.700503 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4ad081c4-10b4-464a-aecc-66f31e728e67 None None] MFA Rules not processed for user `855c2d0dfe92427dae33dd5615d9106f`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:06:44.701824 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4ad081c4-10b4-464a-aecc-66f31e728e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:44.778394 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1389/2779] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:06:44 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:06:44.790659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341420d4-8b1b-43d4-8a96-bc5dfd4aae91 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:44.792921 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-341420d4-8b1b-43d4-8a96-bc5dfd4aae91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:44.857295 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341420d4-8b1b-43d4-8a96-bc5dfd4aae91 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:44.861043 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00bc46-0eb7-4381-bf33-0079f768a92b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:44.861697 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00bc46-0eb7-4381-bf33-0079f768a92b service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:44.861989 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00bc46-0eb7-4381-bf33-0079f768a92b service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:44.862788 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00bc46-0eb7-4381-bf33-0079f768a92b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:44.929358 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e00bc46-0eb7-4381-bf33-0079f768a92b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:44.930717 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e00bc46-0eb7-4381-bf33-0079f768a92b service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:44.942777 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1391/2780] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:06:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:06:47.543937 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad081c4-10b4-464a-aecc-66f31e728e67 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:47.622651 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad081c4-10b4-464a-aecc-66f31e728e67 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:47.628249 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d19ba-5abf-4475-a650-d786d043fc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:47.629135 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d19ba-5abf-4475-a650-d786d043fc95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:47.629414 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d19ba-5abf-4475-a650-d786d043fc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:47.629688 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d19ba-5abf-4475-a650-d786d043fc95 admin admin] PATH_INFO: `/v3/users/17609ecb08c14f20992c7214d64c8c15` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:47.631494 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d19ba-5abf-4475-a650-d786d043fc95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17609ecb08c14f20992c7214d64c8c15)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:47.633451 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d19ba-5abf-4475-a650-d786d043fc95 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:47.657520 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:06:47.685667 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-101d19ba-5abf-4475-a650-d786d043fc95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17609ecb08c14f20992c7214d64c8c15'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:47.694631 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-101d19ba-5abf-4475-a650-d786d043fc95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17609ecb08c14f20992c7214d64c8c15'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:47.702765 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-101d19ba-5abf-4475-a650-d786d043fc95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17609ecb08c14f20992c7214d64c8c15'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:47.703452 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-101d19ba-5abf-4475-a650-d786d043fc95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17609ecb08c14f20992c7214d64c8c15'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:47.704108 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-101d19ba-5abf-4475-a650-d786d043fc95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17609ecb08c14f20992c7214d64c8c15'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:47.726757 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1390/2781] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:06:47 2022] DELETE /identity/v3/users/17609ecb08c14f20992c7214d64c8c15 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:06:47.733952 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b41e98-f14c-46cc-9522-fea57f9a2614 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:47.734478 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b41e98-f14c-46cc-9522-fea57f9a2614 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:47.734568 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b41e98-f14c-46cc-9522-fea57f9a2614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:47.755735 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-83b41e98-f14c-46cc-9522-fea57f9a2614 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:06:47.756983 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-83b41e98-f14c-46cc-9522-fea57f9a2614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:47.800957 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:06:47.823340 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1392/2782] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:06:47.835314 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d19ba-5abf-4475-a650-d786d043fc95 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:47.836806 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-101d19ba-5abf-4475-a650-d786d043fc95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:47.901583 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d19ba-5abf-4475-a650-d786d043fc95 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:47.906658 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5818cb0-318b-44d3-bafb-3f089ed2d8b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:47.907331 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5818cb0-318b-44d3-bafb-3f089ed2d8b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:47.907833 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5818cb0-318b-44d3-bafb-3f089ed2d8b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:47.908254 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5818cb0-318b-44d3-bafb-3f089ed2d8b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:47.920743 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5818cb0-318b-44d3-bafb-3f089ed2d8b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:47.922202 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5818cb0-318b-44d3-bafb-3f089ed2d8b0 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:47.933305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1391/2783] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:06:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:06:48.256872 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b41e98-f14c-46cc-9522-fea57f9a2614 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:48.263266 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-83b41e98-f14c-46cc-9522-fea57f9a2614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:48.281902 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b41e98-f14c-46cc-9522-fea57f9a2614 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:48.290499 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c0e8a0-984c-40ab-9aa8-10e68179bc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:48.292345 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0e8a0-984c-40ab-9aa8-10e68179bc99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:48.292784 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0e8a0-984c-40ab-9aa8-10e68179bc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:48.293175 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0e8a0-984c-40ab-9aa8-10e68179bc99 admin admin] PATH_INFO: `/v3/projects/5b795671c12b4ac1817149c78575f577` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:48.295100 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c0e8a0-984c-40ab-9aa8-10e68179bc99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b795671c12b4ac1817149c78575f577)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:48.297492 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c0e8a0-984c-40ab-9aa8-10e68179bc99 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:48.364464 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a2c0e8a0-984c-40ab-9aa8-10e68179bc99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b795671c12b4ac1817149c78575f577', 'request_id': 'req-a2c0e8a0-984c-40ab-9aa8-10e68179bc99'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:48.375789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a2c0e8a0-984c-40ab-9aa8-10e68179bc99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b795671c12b4ac1817149c78575f577', 'request_id': 'req-a2c0e8a0-984c-40ab-9aa8-10e68179bc99'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:48.385012 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a2c0e8a0-984c-40ab-9aa8-10e68179bc99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b795671c12b4ac1817149c78575f577', 'request_id': 'req-a2c0e8a0-984c-40ab-9aa8-10e68179bc99'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:48.397970 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a2c0e8a0-984c-40ab-9aa8-10e68179bc99 admin admin] The token cache is being invalidate because project 5b795671c12b4ac1817149c78575f577 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:06:48.398257 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a2c0e8a0-984c-40ab-9aa8-10e68179bc99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:48.398982 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a2c0e8a0-984c-40ab-9aa8-10e68179bc99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:48.400729 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1393/2784] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:06:48 2022] DELETE /identity/v3/projects/5b795671c12b4ac1817149c78575f577 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:06:48.405191 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5818cb0-318b-44d3-bafb-3f089ed2d8b0 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:48.416149 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5818cb0-318b-44d3-bafb-3f089ed2d8b0 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:48.482994 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d6c04-1da8-4c28-9e87-0a7376a80f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:48.483956 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d6c04-1da8-4c28-9e87-0a7376a80f15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:48.484260 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d6c04-1da8-4c28-9e87-0a7376a80f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:48.484537 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d6c04-1da8-4c28-9e87-0a7376a80f15 admin admin] PATH_INFO: `/v3/users/855c2d0dfe92427dae33dd5615d9106f` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:48.485836 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51d6c04-1da8-4c28-9e87-0a7376a80f15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=855c2d0dfe92427dae33dd5615d9106f)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:48.488240 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51d6c04-1da8-4c28-9e87-0a7376a80f15 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:48.543690 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:06:48.546768 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c51d6c04-1da8-4c28-9e87-0a7376a80f15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '855c2d0dfe92427dae33dd5615d9106f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:48.558151 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c51d6c04-1da8-4c28-9e87-0a7376a80f15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '855c2d0dfe92427dae33dd5615d9106f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:48.567104 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c51d6c04-1da8-4c28-9e87-0a7376a80f15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '855c2d0dfe92427dae33dd5615d9106f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:48.567787 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c51d6c04-1da8-4c28-9e87-0a7376a80f15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '855c2d0dfe92427dae33dd5615d9106f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:48.568433 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c51d6c04-1da8-4c28-9e87-0a7376a80f15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '855c2d0dfe92427dae33dd5615d9106f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:48.589652 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1392/2785] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:06:48 2022] DELETE /identity/v3/users/855c2d0dfe92427dae33dd5615d9106f => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:06:48.917602 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c0e8a0-984c-40ab-9aa8-10e68179bc99 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:48.921151 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a2c0e8a0-984c-40ab-9aa8-10e68179bc99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:48.939067 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c0e8a0-984c-40ab-9aa8-10e68179bc99 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:49.010438 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2e0db-1feb-4e8c-9162-aeacb5fd498d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:49.011528 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2e0db-1feb-4e8c-9162-aeacb5fd498d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:49.011939 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2e0db-1feb-4e8c-9162-aeacb5fd498d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:49.012314 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2e0db-1feb-4e8c-9162-aeacb5fd498d admin admin] PATH_INFO: `/v3/projects/2ffe2ead096c48b78b42b0deecc3fb69` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:49.013832 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd2e0db-1feb-4e8c-9162-aeacb5fd498d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ffe2ead096c48b78b42b0deecc3fb69)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:49.016141 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd2e0db-1feb-4e8c-9162-aeacb5fd498d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:49.080070 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9bd2e0db-1feb-4e8c-9162-aeacb5fd498d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ffe2ead096c48b78b42b0deecc3fb69', 'request_id': 'req-9bd2e0db-1feb-4e8c-9162-aeacb5fd498d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:49.090573 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9bd2e0db-1feb-4e8c-9162-aeacb5fd498d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ffe2ead096c48b78b42b0deecc3fb69', 'request_id': 'req-9bd2e0db-1feb-4e8c-9162-aeacb5fd498d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:49.098425 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9bd2e0db-1feb-4e8c-9162-aeacb5fd498d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ffe2ead096c48b78b42b0deecc3fb69', 'request_id': 'req-9bd2e0db-1feb-4e8c-9162-aeacb5fd498d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:49.107862 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:06:49.114381 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9bd2e0db-1feb-4e8c-9162-aeacb5fd498d admin admin] The token cache is being invalidate because project 2ffe2ead096c48b78b42b0deecc3fb69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:06:49.114712 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9bd2e0db-1feb-4e8c-9162-aeacb5fd498d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:49.119129 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9bd2e0db-1feb-4e8c-9162-aeacb5fd498d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:06:49.122452 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1394/2786] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:06:48 2022] DELETE /identity/v3/projects/2ffe2ead096c48b78b42b0deecc3fb69 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 20:06:49.131171 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d913ab7-221e-44a5-b3a0-153fe91fb96f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:49.131626 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d913ab7-221e-44a5-b3a0-153fe91fb96f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:49.131999 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d913ab7-221e-44a5-b3a0-153fe91fb96f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:49.160533 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3d913ab7-221e-44a5-b3a0-153fe91fb96f None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:06:49.162837 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3d913ab7-221e-44a5-b3a0-153fe91fb96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:49.246583 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1393/2787] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 20:06:49.254865 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b81b4-8e92-4498-9b22-1296c21dcc29 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:49.255221 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b81b4-8e92-4498-9b22-1296c21dcc29 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:49.255659 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b81b4-8e92-4498-9b22-1296c21dcc29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:49.280465 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b78b81b4-8e92-4498-9b22-1296c21dcc29 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:06:49.281744 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b78b81b4-8e92-4498-9b22-1296c21dcc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:49.292935 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1395/2788] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:06:49.297382 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d913ab7-221e-44a5-b3a0-153fe91fb96f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:49.311671 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d913ab7-221e-44a5-b3a0-153fe91fb96f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:49.319677 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e66969-9d71-44e8-83ae-b3930a1b0fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:49.321454 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e66969-9d71-44e8-83ae-b3930a1b0fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:49.321818 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e66969-9d71-44e8-83ae-b3930a1b0fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:49.322256 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e66969-9d71-44e8-83ae-b3930a1b0fc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:49.324415 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e66969-9d71-44e8-83ae-b3930a1b0fc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 20:06:49.324974 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e66969-9d71-44e8-83ae-b3930a1b0fc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:49.327219 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e66969-9d71-44e8-83ae-b3930a1b0fc3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:49.345017 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1394/2789] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 20:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 20:06:49.351466 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78b81b4-8e92-4498-9b22-1296c21dcc29 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:49.361858 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78b81b4-8e92-4498-9b22-1296c21dcc29 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:49.367459 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae0169-cac0-4f73-b407-ea7b3b0fa885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:49.368443 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae0169-cac0-4f73-b407-ea7b3b0fa885 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:49.368721 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae0169-cac0-4f73-b407-ea7b3b0fa885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:49.368999 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae0169-cac0-4f73-b407-ea7b3b0fa885 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:49.369687 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bae0169-cac0-4f73-b407-ea7b3b0fa885 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:49.371643 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bae0169-cac0-4f73-b407-ea7b3b0fa885 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:49.401115 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1396/2790] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:06:49 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:06:49.405480 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e66969-9d71-44e8-83ae-b3930a1b0fc3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:49.417726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e66969-9d71-44e8-83ae-b3930a1b0fc3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:49.492323 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6970c46-135e-4076-9f89-bb27e130977f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:49.494224 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6970c46-135e-4076-9f89-bb27e130977f admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:49.494539 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6970c46-135e-4076-9f89-bb27e130977f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:49.494839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6970c46-135e-4076-9f89-bb27e130977f admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:49.495591 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6970c46-135e-4076-9f89-bb27e130977f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:49.497587 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6970c46-135e-4076-9f89-bb27e130977f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:49.521115 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1395/2791] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:06:49 2022] POST /identity/v3/users => generated 284 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:06:49.525690 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae0169-cac0-4f73-b407-ea7b3b0fa885 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:49.541427 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae0169-cac0-4f73-b407-ea7b3b0fa885 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:49.546738 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9195ad4-bc96-43df-b2ac-3f94bfed9a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:49.547425 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9195ad4-bc96-43df-b2ac-3f94bfed9a77 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:49.547742 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9195ad4-bc96-43df-b2ac-3f94bfed9a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:49.548016 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9195ad4-bc96-43df-b2ac-3f94bfed9a77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:49.548765 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9195ad4-bc96-43df-b2ac-3f94bfed9a77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:49.551065 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9195ad4-bc96-43df-b2ac-3f94bfed9a77 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:49.563339 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1397/2792] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:06:49.568602 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6970c46-135e-4076-9f89-bb27e130977f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:49.582129 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6970c46-135e-4076-9f89-bb27e130977f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:49.590228 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4664a627-0634-4bb1-968c-6bec5c4e45d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:49.590929 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664a627-0634-4bb1-968c-6bec5c4e45d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:49.591201 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664a627-0634-4bb1-968c-6bec5c4e45d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:49.591484 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664a627-0634-4bb1-968c-6bec5c4e45d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:49.592380 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4664a627-0634-4bb1-968c-6bec5c4e45d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:49.594028 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4664a627-0634-4bb1-968c-6bec5c4e45d1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:49.613318 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1396/2793] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:06:49.618561 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9195ad4-bc96-43df-b2ac-3f94bfed9a77 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:49.629531 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9195ad4-bc96-43df-b2ac-3f94bfed9a77 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:49.635204 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de549d7-afe2-4a3f-aaee-f3fdbd914dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:49.636452 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de549d7-afe2-4a3f-aaee-f3fdbd914dbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:49.636787 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de549d7-afe2-4a3f-aaee-f3fdbd914dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:49.637110 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de549d7-afe2-4a3f-aaee-f3fdbd914dbc admin admin] PATH_INFO: `/v3/projects/af99e2a06e684dda915085b9576b3df8/users/4a418b7f97e341c3a39c20a0dc01afde/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:49.655693 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de549d7-afe2-4a3f-aaee-f3fdbd914dbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=af99e2a06e684dda915085b9576b3df8, user_id=4a418b7f97e341c3a39c20a0dc01afde, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:49.658495 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de549d7-afe2-4a3f-aaee-f3fdbd914dbc admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:49.674434 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1398/2794] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:06:49 2022] PUT /identity/v3/projects/af99e2a06e684dda915085b9576b3df8/users/4a418b7f97e341c3a39c20a0dc01afde/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:06:49.686570 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccdd3da-0e81-46fb-b1e2-76a6af749360 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:49.686871 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccdd3da-0e81-46fb-b1e2-76a6af749360 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:49.687150 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccdd3da-0e81-46fb-b1e2-76a6af749360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:49.725640 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-cccdd3da-0e81-46fb-b1e2-76a6af749360 None None] MFA Rules not processed for user `4a418b7f97e341c3a39c20a0dc01afde`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:06:49.727475 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cccdd3da-0e81-46fb-b1e2-76a6af749360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:49.755614 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:06:49.830975 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1397/2795] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:06:49 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:06:49.843775 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de549d7-afe2-4a3f-aaee-f3fdbd914dbc admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:06:49.845809 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5de549d7-afe2-4a3f-aaee-f3fdbd914dbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:06:49.918250 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de549d7-afe2-4a3f-aaee-f3fdbd914dbc admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:06:49.922607 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d771ec37-a49a-4934-98eb-63bde28f9402 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98560723b49940d38a6a5e9cf3f5d3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:06:49.923377 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771ec37-a49a-4934-98eb-63bde28f9402 service glance] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:06:49.923727 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771ec37-a49a-4934-98eb-63bde28f9402 service glance] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:06:49.924114 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771ec37-a49a-4934-98eb-63bde28f9402 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:06:49.934755 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d771ec37-a49a-4934-98eb-63bde28f9402 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:06:49.936246 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d771ec37-a49a-4934-98eb-63bde28f9402 service glance] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:06:49.947400 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1399/2796] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:06:49 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:06:59.827422 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=20 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:06:59.942153 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=54,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:07:26.581388 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccdd3da-0e81-46fb-b1e2-76a6af749360 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:26.656257 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccdd3da-0e81-46fb-b1e2-76a6af749360 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:26.660881 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b916768-cfff-4f7a-9f7f-d00a3708417f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:26.661788 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b916768-cfff-4f7a-9f7f-d00a3708417f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:26.662071 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b916768-cfff-4f7a-9f7f-d00a3708417f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:26.662375 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b916768-cfff-4f7a-9f7f-d00a3708417f admin admin] PATH_INFO: `/v3/users/4a418b7f97e341c3a39c20a0dc01afde` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:26.663612 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b916768-cfff-4f7a-9f7f-d00a3708417f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a418b7f97e341c3a39c20a0dc01afde)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:26.670044 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b916768-cfff-4f7a-9f7f-d00a3708417f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:26.720769 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6b916768-cfff-4f7a-9f7f-d00a3708417f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a418b7f97e341c3a39c20a0dc01afde'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:07:26.729061 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6b916768-cfff-4f7a-9f7f-d00a3708417f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a418b7f97e341c3a39c20a0dc01afde'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:07:26.736941 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6b916768-cfff-4f7a-9f7f-d00a3708417f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a418b7f97e341c3a39c20a0dc01afde'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:07:26.737790 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6b916768-cfff-4f7a-9f7f-d00a3708417f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a418b7f97e341c3a39c20a0dc01afde'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:07:26.738402 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6b916768-cfff-4f7a-9f7f-d00a3708417f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a418b7f97e341c3a39c20a0dc01afde'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:07:26.757721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1398/2797] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:07:26 2022] DELETE /identity/v3/users/4a418b7f97e341c3a39c20a0dc01afde => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:07:26.763162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77e217-b80f-47f9-8f99-b1091e53be4b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:26.763755 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77e217-b80f-47f9-8f99-b1091e53be4b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:26.763845 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77e217-b80f-47f9-8f99-b1091e53be4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:26.784260 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6a77e217-b80f-47f9-8f99-b1091e53be4b None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:07:26.785480 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6a77e217-b80f-47f9-8f99-b1091e53be4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:07:26.852388 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1400/2798] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:07:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 20:07:26.862794 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b916768-cfff-4f7a-9f7f-d00a3708417f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:26.864265 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6b916768-cfff-4f7a-9f7f-d00a3708417f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:07:26.936394 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b916768-cfff-4f7a-9f7f-d00a3708417f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:26.940186 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a17f85-3c85-4224-b975-b5b0b4024cd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:26.940849 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a17f85-3c85-4224-b975-b5b0b4024cd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:26.941136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a17f85-3c85-4224-b975-b5b0b4024cd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:26.941415 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a17f85-3c85-4224-b975-b5b0b4024cd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:26.953334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a17f85-3c85-4224-b975-b5b0b4024cd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:26.955072 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a17f85-3c85-4224-b975-b5b0b4024cd8 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:26.966560 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1399/2799] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:07:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:07:27.260047 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a77e217-b80f-47f9-8f99-b1091e53be4b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:27.262833 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6a77e217-b80f-47f9-8f99-b1091e53be4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:07:27.276524 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a77e217-b80f-47f9-8f99-b1091e53be4b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:27.283900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3af24-4f3c-4de3-bc60-f9e284c240af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:27.284373 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3af24-4f3c-4de3-bc60-f9e284c240af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:27.284663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3af24-4f3c-4de3-bc60-f9e284c240af admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:27.284939 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3af24-4f3c-4de3-bc60-f9e284c240af admin admin] PATH_INFO: `/v3/projects/af99e2a06e684dda915085b9576b3df8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:27.286191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a3af24-4f3c-4de3-bc60-f9e284c240af admin admin] RBAC: Authorizing `identity:delete_project(project_id=af99e2a06e684dda915085b9576b3df8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:27.288687 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a3af24-4f3c-4de3-bc60-f9e284c240af admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:27.349943 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-38a3af24-4f3c-4de3-bc60-f9e284c240af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af99e2a06e684dda915085b9576b3df8', 'request_id': 'req-38a3af24-4f3c-4de3-bc60-f9e284c240af'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:07:27.358412 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-38a3af24-4f3c-4de3-bc60-f9e284c240af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af99e2a06e684dda915085b9576b3df8', 'request_id': 'req-38a3af24-4f3c-4de3-bc60-f9e284c240af'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:07:27.366205 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-38a3af24-4f3c-4de3-bc60-f9e284c240af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af99e2a06e684dda915085b9576b3df8', 'request_id': 'req-38a3af24-4f3c-4de3-bc60-f9e284c240af'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:07:27.379089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-38a3af24-4f3c-4de3-bc60-f9e284c240af admin admin] The token cache is being invalidate because project af99e2a06e684dda915085b9576b3df8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:07:27.379384 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-38a3af24-4f3c-4de3-bc60-f9e284c240af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:07:27.380126 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-38a3af24-4f3c-4de3-bc60-f9e284c240af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:07:27.381703 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1401/2800] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:07:27 2022] DELETE /identity/v3/projects/af99e2a06e684dda915085b9576b3df8 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:07:27.387784 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289e6cf-cf5d-4ad6-885d-546ad7820bcf None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:27.388080 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289e6cf-cf5d-4ad6-885d-546ad7820bcf None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:27.388362 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289e6cf-cf5d-4ad6-885d-546ad7820bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:27.410216 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f289e6cf-cf5d-4ad6-885d-546ad7820bcf None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:07:27.411460 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f289e6cf-cf5d-4ad6-885d-546ad7820bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:07:27.431090 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:07:27.477837 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1400/2801] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:07:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:07:27.487100 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f30c0c-f468-4f52-9ff0-47af396d6260 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:27.487394 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f30c0c-f468-4f52-9ff0-47af396d6260 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:27.487713 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f30c0c-f468-4f52-9ff0-47af396d6260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:27.509009 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-14f30c0c-f468-4f52-9ff0-47af396d6260 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:07:27.510319 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-14f30c0c-f468-4f52-9ff0-47af396d6260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:07:27.518356 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1402/2802] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:07:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:07:27.522815 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f289e6cf-cf5d-4ad6-885d-546ad7820bcf None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:27.532551 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f289e6cf-cf5d-4ad6-885d-546ad7820bcf None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:27.537182 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05653f18-467b-40ac-89fc-bb9ef520190d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:27.537836 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05653f18-467b-40ac-89fc-bb9ef520190d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:27.538118 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05653f18-467b-40ac-89fc-bb9ef520190d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:27.538394 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05653f18-467b-40ac-89fc-bb9ef520190d admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:27.539113 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05653f18-467b-40ac-89fc-bb9ef520190d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 20:07:27.539415 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05653f18-467b-40ac-89fc-bb9ef520190d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:27.541038 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05653f18-467b-40ac-89fc-bb9ef520190d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:27.552501 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1401/2803] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 20:07:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 20:07:27.557481 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f30c0c-f468-4f52-9ff0-47af396d6260 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:27.566911 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f30c0c-f468-4f52-9ff0-47af396d6260 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:27.572285 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660bce22-1f9c-4707-9b35-08ac1152c41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:27.572939 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bce22-1f9c-4707-9b35-08ac1152c41d admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:27.573226 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bce22-1f9c-4707-9b35-08ac1152c41d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:27.573505 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bce22-1f9c-4707-9b35-08ac1152c41d admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:27.574201 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660bce22-1f9c-4707-9b35-08ac1152c41d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:27.576145 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660bce22-1f9c-4707-9b35-08ac1152c41d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:27.606579 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1403/2804] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:07:27 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:07:27.610763 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05653f18-467b-40ac-89fc-bb9ef520190d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:27.621444 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05653f18-467b-40ac-89fc-bb9ef520190d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:27.686446 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c100e9-d88a-416a-aece-c40c6692d89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:27.687130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c100e9-d88a-416a-aece-c40c6692d89c admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:27.687418 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c100e9-d88a-416a-aece-c40c6692d89c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:27.687727 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c100e9-d88a-416a-aece-c40c6692d89c admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:27.688416 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c100e9-d88a-416a-aece-c40c6692d89c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:27.690363 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c100e9-d88a-416a-aece-c40c6692d89c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:27.713604 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1402/2805] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:07:27 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:07:27.717691 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660bce22-1f9c-4707-9b35-08ac1152c41d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:27.728351 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660bce22-1f9c-4707-9b35-08ac1152c41d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:27.733025 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369336d3-a9dd-44cd-a1d7-3720ee3f4b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:27.733684 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369336d3-a9dd-44cd-a1d7-3720ee3f4b54 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:27.733967 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369336d3-a9dd-44cd-a1d7-3720ee3f4b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:27.734297 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369336d3-a9dd-44cd-a1d7-3720ee3f4b54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:27.735049 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369336d3-a9dd-44cd-a1d7-3720ee3f4b54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:27.736674 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369336d3-a9dd-44cd-a1d7-3720ee3f4b54 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:27.747240 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1404/2806] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:07:27.751358 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c100e9-d88a-416a-aece-c40c6692d89c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:27.762129 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c100e9-d88a-416a-aece-c40c6692d89c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:27.767384 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a33b76a-414b-4a42-b716-8ce52aa97fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:27.768096 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33b76a-414b-4a42-b716-8ce52aa97fe6 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:27.768374 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33b76a-414b-4a42-b716-8ce52aa97fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:27.768773 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33b76a-414b-4a42-b716-8ce52aa97fe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:27.769520 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a33b76a-414b-4a42-b716-8ce52aa97fe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:27.771134 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a33b76a-414b-4a42-b716-8ce52aa97fe6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:27.783397 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1403/2807] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:07:27.787836 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369336d3-a9dd-44cd-a1d7-3720ee3f4b54 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:27.798195 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369336d3-a9dd-44cd-a1d7-3720ee3f4b54 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:27.802734 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7759881-baa0-42d1-a4bf-7da0947f9034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:27.803684 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7759881-baa0-42d1-a4bf-7da0947f9034 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:27.803970 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7759881-baa0-42d1-a4bf-7da0947f9034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:27.804250 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7759881-baa0-42d1-a4bf-7da0947f9034 admin admin] PATH_INFO: `/v3/projects/b6986405c7974cc8bdf5d0c470926c1f/users/8143c33eabd74b1b9edc84e8ff155903/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:27.818800 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7759881-baa0-42d1-a4bf-7da0947f9034 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6986405c7974cc8bdf5d0c470926c1f, user_id=8143c33eabd74b1b9edc84e8ff155903, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:27.821985 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7759881-baa0-42d1-a4bf-7da0947f9034 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:27.834931 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1405/2808] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:07:27 2022] PUT /identity/v3/projects/b6986405c7974cc8bdf5d0c470926c1f/users/8143c33eabd74b1b9edc84e8ff155903/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:07:27.841841 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad124db4-0207-437f-a18c-2c5353c4bdc2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:27.842137 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad124db4-0207-437f-a18c-2c5353c4bdc2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:27.842420 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad124db4-0207-437f-a18c-2c5353c4bdc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:27.862318 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ad124db4-0207-437f-a18c-2c5353c4bdc2 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:07:27.863558 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ad124db4-0207-437f-a18c-2c5353c4bdc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:07:27.926779 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1404/2809] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:07:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:07:27.937497 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7759881-baa0-42d1-a4bf-7da0947f9034 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:27.939279 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a7759881-baa0-42d1-a4bf-7da0947f9034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:07:27.962061 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:07:28.000794 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7759881-baa0-42d1-a4bf-7da0947f9034 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:28.004442 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfebdfa2-ffbb-463d-bbd5-405384d9481d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:28.005086 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebdfa2-ffbb-463d-bbd5-405384d9481d service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:28.005370 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebdfa2-ffbb-463d-bbd5-405384d9481d service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:28.005647 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebdfa2-ffbb-463d-bbd5-405384d9481d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:28.014807 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfebdfa2-ffbb-463d-bbd5-405384d9481d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:28.016283 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfebdfa2-ffbb-463d-bbd5-405384d9481d service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:28.028158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1406/2810] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:07:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:07:32.189248 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a08d6-058a-4ff6-9135-bb1be1bab8dd None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:32.189727 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a08d6-058a-4ff6-9135-bb1be1bab8dd None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:32.189816 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a08d6-058a-4ff6-9135-bb1be1bab8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:32.219636 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-675a08d6-058a-4ff6-9135-bb1be1bab8dd None None] MFA Rules not processed for user `8143c33eabd74b1b9edc84e8ff155903`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:07:32.220787 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-675a08d6-058a-4ff6-9135-bb1be1bab8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:07:32.239782 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:07:32.292230 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1405/2811] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:07:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:07:32.296730 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfebdfa2-ffbb-463d-bbd5-405384d9481d service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:32.307609 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfebdfa2-ffbb-463d-bbd5-405384d9481d service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:32.312606 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579173c-08c0-474a-912d-0849f3e60859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:32.313284 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579173c-08c0-474a-912d-0849f3e60859 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:32.313579 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579173c-08c0-474a-912d-0849f3e60859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:32.313863 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579173c-08c0-474a-912d-0849f3e60859 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:32.314576 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5579173c-08c0-474a-912d-0849f3e60859 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:32.316583 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5579173c-08c0-474a-912d-0849f3e60859 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:32.359862 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1407/2812] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:07:32 2022] POST /identity/v3/projects => generated 379 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:07:32.361082 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a08d6-058a-4ff6-9135-bb1be1bab8dd None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:32.375203 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a08d6-058a-4ff6-9135-bb1be1bab8dd None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:32.438026 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71abb8af-5845-4b5f-9233-5fc2d3ea5540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:32.438710 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71abb8af-5845-4b5f-9233-5fc2d3ea5540 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:32.438992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71abb8af-5845-4b5f-9233-5fc2d3ea5540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:32.439270 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71abb8af-5845-4b5f-9233-5fc2d3ea5540 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:32.440021 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71abb8af-5845-4b5f-9233-5fc2d3ea5540 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:32.441920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71abb8af-5845-4b5f-9233-5fc2d3ea5540 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:32.464722 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1406/2813] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:07:32 2022] POST /identity/v3/users => generated 296 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:07:32.468752 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579173c-08c0-474a-912d-0849f3e60859 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:32.479807 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579173c-08c0-474a-912d-0849f3e60859 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:32.484297 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c66c358-a0bc-483b-8abe-d31bd6cb2d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:32.484981 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66c358-a0bc-483b-8abe-d31bd6cb2d97 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:32.485264 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66c358-a0bc-483b-8abe-d31bd6cb2d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:32.485537 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66c358-a0bc-483b-8abe-d31bd6cb2d97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:32.486265 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c66c358-a0bc-483b-8abe-d31bd6cb2d97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:32.488018 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c66c358-a0bc-483b-8abe-d31bd6cb2d97 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:32.500121 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1408/2814] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:07:32.504371 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71abb8af-5845-4b5f-9233-5fc2d3ea5540 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:32.515485 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71abb8af-5845-4b5f-9233-5fc2d3ea5540 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:32.520347 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e8e3e-275a-4b96-b6f8-66ac93a043c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:32.521280 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e8e3e-275a-4b96-b6f8-66ac93a043c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:32.521565 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e8e3e-275a-4b96-b6f8-66ac93a043c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:32.521839 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e8e3e-275a-4b96-b6f8-66ac93a043c0 admin admin] PATH_INFO: `/v3/projects/fd04bb0df5a44c98a8dd34d746ddb6d9/users/b081268a6421470683561d5b6b743ac9/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:32.537419 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1e8e3e-275a-4b96-b6f8-66ac93a043c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd04bb0df5a44c98a8dd34d746ddb6d9, user_id=b081268a6421470683561d5b6b743ac9, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:32.540396 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1e8e3e-275a-4b96-b6f8-66ac93a043c0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:32.555179 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1407/2815] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:07:32 2022] PUT /identity/v3/projects/fd04bb0df5a44c98a8dd34d746ddb6d9/users/b081268a6421470683561d5b6b743ac9/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:07:32.559332 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c66c358-a0bc-483b-8abe-d31bd6cb2d97 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:32.569388 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c66c358-a0bc-483b-8abe-d31bd6cb2d97 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:32.626303 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73408547-e41d-4c79-9112-979e17683ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:32.627006 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73408547-e41d-4c79-9112-979e17683ac1 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:32.627289 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73408547-e41d-4c79-9112-979e17683ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:32.627673 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73408547-e41d-4c79-9112-979e17683ac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:32.628410 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73408547-e41d-4c79-9112-979e17683ac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:32.630007 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73408547-e41d-4c79-9112-979e17683ac1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:32.640548 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1409/2816] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:07:32.644692 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e8e3e-275a-4b96-b6f8-66ac93a043c0 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:32.655061 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e8e3e-275a-4b96-b6f8-66ac93a043c0 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:32.659833 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e0b4c-249f-4c6a-8ae8-df776ee7ae0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:32.660745 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e0b4c-249f-4c6a-8ae8-df776ee7ae0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:32.661025 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e0b4c-249f-4c6a-8ae8-df776ee7ae0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:32.661298 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e0b4c-249f-4c6a-8ae8-df776ee7ae0e admin admin] PATH_INFO: `/v3/projects/fd04bb0df5a44c98a8dd34d746ddb6d9/users/b081268a6421470683561d5b6b743ac9/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:32.663296 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e0b4c-249f-4c6a-8ae8-df776ee7ae0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd04bb0df5a44c98a8dd34d746ddb6d9, user_id=b081268a6421470683561d5b6b743ac9, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:32.666309 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e0b4c-249f-4c6a-8ae8-df776ee7ae0e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:32.678278 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1408/2817] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:07:32 2022] PUT /identity/v3/projects/fd04bb0df5a44c98a8dd34d746ddb6d9/users/b081268a6421470683561d5b6b743ac9/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:07:37.499009 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85237ed2-51d7-4a39-bff8-18bb94557c1e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:37.499498 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85237ed2-51d7-4a39-bff8-18bb94557c1e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:37.499677 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85237ed2-51d7-4a39-bff8-18bb94557c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:37.531836 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-85237ed2-51d7-4a39-bff8-18bb94557c1e None None] MFA Rules not processed for user `b081268a6421470683561d5b6b743ac9`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:07:37.533076 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-85237ed2-51d7-4a39-bff8-18bb94557c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:07:37.627716 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1410/2818] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:07:37 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:07:37.643983 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e0b4c-249f-4c6a-8ae8-df776ee7ae0e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:37.646273 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-073e0b4c-249f-4c6a-8ae8-df776ee7ae0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:07:37.729296 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:07:37.763081 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e0b4c-249f-4c6a-8ae8-df776ee7ae0e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:37.778597 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed85d0d8-e66a-42f4-80a0-21d9cf7b9dd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:37.779136 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85d0d8-e66a-42f4-80a0-21d9cf7b9dd7 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:37.779424 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85d0d8-e66a-42f4-80a0-21d9cf7b9dd7 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:37.779740 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85d0d8-e66a-42f4-80a0-21d9cf7b9dd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:37.863326 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed85d0d8-e66a-42f4-80a0-21d9cf7b9dd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:37.871116 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed85d0d8-e66a-42f4-80a0-21d9cf7b9dd7 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:37.876890 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85237ed2-51d7-4a39-bff8-18bb94557c1e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:37.881895 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1409/2819] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:07:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:07:37.902576 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:07:37.953216 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85237ed2-51d7-4a39-bff8-18bb94557c1e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:37.957358 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da299f-a2d2-47e5-9c25-247f955883b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:37.957986 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da299f-a2d2-47e5-9c25-247f955883b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:37.958216 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da299f-a2d2-47e5-9c25-247f955883b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:37.958497 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da299f-a2d2-47e5-9c25-247f955883b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:37.960489 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-91da299f-a2d2-47e5-9c25-247f955883b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:07:38.029834 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da299f-a2d2-47e5-9c25-247f955883b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:38.031329 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da299f-a2d2-47e5-9c25-247f955883b4 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:38.043203 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1411/2820] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:07:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 20:07:42.331329 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed85d0d8-e66a-42f4-80a0-21d9cf7b9dd7 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:42.332959 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ed85d0d8-e66a-42f4-80a0-21d9cf7b9dd7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:07:42.352074 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed85d0d8-e66a-42f4-80a0-21d9cf7b9dd7 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:42.356832 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d02e48-7d0a-437d-82a2-b03d10ade5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:42.357713 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d02e48-7d0a-437d-82a2-b03d10ade5e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:42.358001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d02e48-7d0a-437d-82a2-b03d10ade5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:42.358281 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d02e48-7d0a-437d-82a2-b03d10ade5e1 admin admin] PATH_INFO: `/v3/users/6ecf34ceee294b53b5be7c1723b42950` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:42.359537 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d02e48-7d0a-437d-82a2-b03d10ade5e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ecf34ceee294b53b5be7c1723b42950)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:42.361481 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d02e48-7d0a-437d-82a2-b03d10ade5e1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:42.435112 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-38d02e48-7d0a-437d-82a2-b03d10ade5e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ecf34ceee294b53b5be7c1723b42950'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:07:42.447384 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-38d02e48-7d0a-437d-82a2-b03d10ade5e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ecf34ceee294b53b5be7c1723b42950'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:07:42.456617 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-38d02e48-7d0a-437d-82a2-b03d10ade5e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ecf34ceee294b53b5be7c1723b42950'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:07:42.457257 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-38d02e48-7d0a-437d-82a2-b03d10ade5e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ecf34ceee294b53b5be7c1723b42950'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:07:42.457872 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-38d02e48-7d0a-437d-82a2-b03d10ade5e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ecf34ceee294b53b5be7c1723b42950'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:07:42.481814 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1410/2821] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:07:42 2022] DELETE /identity/v3/users/6ecf34ceee294b53b5be7c1723b42950 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:07:42.721888 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da299f-a2d2-47e5-9c25-247f955883b4 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:42.736876 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da299f-a2d2-47e5-9c25-247f955883b4 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:42.834120 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f960273-3b2e-4de3-9554-b32194622094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:42.835871 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f960273-3b2e-4de3-9554-b32194622094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:42.837071 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f960273-3b2e-4de3-9554-b32194622094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:42.837581 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f960273-3b2e-4de3-9554-b32194622094 admin admin] PATH_INFO: `/v3/projects/8185baa0f3e3465f813c0e2dc29ad847` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:42.839137 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f960273-3b2e-4de3-9554-b32194622094 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8185baa0f3e3465f813c0e2dc29ad847)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:42.841400 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f960273-3b2e-4de3-9554-b32194622094 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:42.909427 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1f960273-3b2e-4de3-9554-b32194622094 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8185baa0f3e3465f813c0e2dc29ad847', 'request_id': 'req-1f960273-3b2e-4de3-9554-b32194622094'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:07:42.923153 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1f960273-3b2e-4de3-9554-b32194622094 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8185baa0f3e3465f813c0e2dc29ad847', 'request_id': 'req-1f960273-3b2e-4de3-9554-b32194622094'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:07:42.931585 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:07:42.933236 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1f960273-3b2e-4de3-9554-b32194622094 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8185baa0f3e3465f813c0e2dc29ad847', 'request_id': 'req-1f960273-3b2e-4de3-9554-b32194622094'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:07:42.952048 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1f960273-3b2e-4de3-9554-b32194622094 admin admin] The token cache is being invalidate because project 8185baa0f3e3465f813c0e2dc29ad847 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:07:42.952341 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1f960273-3b2e-4de3-9554-b32194622094 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:07:42.953033 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1f960273-3b2e-4de3-9554-b32194622094 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:07:42.954614 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1412/2822] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:07:42 2022] DELETE /identity/v3/projects/8185baa0f3e3465f813c0e2dc29ad847 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:07:42.961593 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474adf4f-f99e-4c5b-aa25-dd86ccc4fb67 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:42.962110 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474adf4f-f99e-4c5b-aa25-dd86ccc4fb67 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:42.962407 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474adf4f-f99e-4c5b-aa25-dd86ccc4fb67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:42.988756 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-474adf4f-f99e-4c5b-aa25-dd86ccc4fb67 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:07:42.990305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-474adf4f-f99e-4c5b-aa25-dd86ccc4fb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:07:43.073819 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:07:43.083599 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1411/2823] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:07:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 20:07:43.092518 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733fd34-f52a-4187-8e6e-72b9f57f8a09 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:43.092829 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733fd34-f52a-4187-8e6e-72b9f57f8a09 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:43.093111 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733fd34-f52a-4187-8e6e-72b9f57f8a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:43.117585 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3733fd34-f52a-4187-8e6e-72b9f57f8a09 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:07:43.119185 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3733fd34-f52a-4187-8e6e-72b9f57f8a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:07:43.127552 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1413/2824] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:07:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:07:43.131780 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474adf4f-f99e-4c5b-aa25-dd86ccc4fb67 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:43.142041 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474adf4f-f99e-4c5b-aa25-dd86ccc4fb67 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:43.147246 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ab9e8-8c5b-46b7-8f28-7d9edaf177e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:43.148102 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ab9e8-8c5b-46b7-8f28-7d9edaf177e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:43.148388 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ab9e8-8c5b-46b7-8f28-7d9edaf177e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:43.148662 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ab9e8-8c5b-46b7-8f28-7d9edaf177e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:43.149411 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12ab9e8-8c5b-46b7-8f28-7d9edaf177e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 20:07:43.149843 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12ab9e8-8c5b-46b7-8f28-7d9edaf177e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:43.151883 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12ab9e8-8c5b-46b7-8f28-7d9edaf177e3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:43.172231 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1412/2825] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 20:07:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 20:07:43.176675 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3733fd34-f52a-4187-8e6e-72b9f57f8a09 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:43.186972 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3733fd34-f52a-4187-8e6e-72b9f57f8a09 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:43.191702 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f68d19-e057-4939-8363-16dfd920e784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:43.192382 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f68d19-e057-4939-8363-16dfd920e784 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:43.192639 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f68d19-e057-4939-8363-16dfd920e784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:43.192917 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f68d19-e057-4939-8363-16dfd920e784 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:43.193652 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f68d19-e057-4939-8363-16dfd920e784 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:43.195645 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f68d19-e057-4939-8363-16dfd920e784 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:43.240615 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1414/2826] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:07:43 2022] POST /identity/v3/projects => generated 383 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:07:43.242334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ab9e8-8c5b-46b7-8f28-7d9edaf177e3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:43.258191 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ab9e8-8c5b-46b7-8f28-7d9edaf177e3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:43.328151 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4986ec1c-4d5d-442c-ba7d-130ed7bd23ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:43.328842 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986ec1c-4d5d-442c-ba7d-130ed7bd23ad admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:43.329125 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986ec1c-4d5d-442c-ba7d-130ed7bd23ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:43.329405 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986ec1c-4d5d-442c-ba7d-130ed7bd23ad admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:43.330104 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4986ec1c-4d5d-442c-ba7d-130ed7bd23ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:43.332082 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4986ec1c-4d5d-442c-ba7d-130ed7bd23ad admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:43.359830 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1413/2827] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:07:43 2022] POST /identity/v3/users => generated 292 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 20:07:43.361279 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f68d19-e057-4939-8363-16dfd920e784 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:43.375367 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f68d19-e057-4939-8363-16dfd920e784 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:43.380813 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723794ec-fae7-40e8-a6dd-62e499339a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:43.381577 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723794ec-fae7-40e8-a6dd-62e499339a8b admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:43.381864 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723794ec-fae7-40e8-a6dd-62e499339a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:43.382168 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723794ec-fae7-40e8-a6dd-62e499339a8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:43.382930 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723794ec-fae7-40e8-a6dd-62e499339a8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:43.384595 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723794ec-fae7-40e8-a6dd-62e499339a8b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:43.400078 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1415/2828] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:07:43.400992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4986ec1c-4d5d-442c-ba7d-130ed7bd23ad admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:43.414335 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4986ec1c-4d5d-442c-ba7d-130ed7bd23ad admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:43.419336 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05533ccc-83b5-4795-bf17-06d6e90a7c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:43.420051 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05533ccc-83b5-4795-bf17-06d6e90a7c32 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:43.420335 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05533ccc-83b5-4795-bf17-06d6e90a7c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:43.420614 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05533ccc-83b5-4795-bf17-06d6e90a7c32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:43.421350 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05533ccc-83b5-4795-bf17-06d6e90a7c32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:43.422959 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05533ccc-83b5-4795-bf17-06d6e90a7c32 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:43.437507 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1414/2829] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:07:43.438692 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723794ec-fae7-40e8-a6dd-62e499339a8b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:43.449892 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723794ec-fae7-40e8-a6dd-62e499339a8b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:43.461257 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57ed4d5-c7c5-4e69-85b5-b67744ee505d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:43.463834 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ed4d5-c7c5-4e69-85b5-b67744ee505d admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:43.464714 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ed4d5-c7c5-4e69-85b5-b67744ee505d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:43.465206 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ed4d5-c7c5-4e69-85b5-b67744ee505d admin admin] PATH_INFO: `/v3/projects/d5748d52c41f4de1b9913d3cb1ffd134/users/38f31b382dea44dc9ae5041e70255082/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:43.487595 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57ed4d5-c7c5-4e69-85b5-b67744ee505d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5748d52c41f4de1b9913d3cb1ffd134, user_id=38f31b382dea44dc9ae5041e70255082, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:43.491013 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57ed4d5-c7c5-4e69-85b5-b67744ee505d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:43.510805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1416/2830] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:07:43 2022] PUT /identity/v3/projects/d5748d52c41f4de1b9913d3cb1ffd134/users/38f31b382dea44dc9ae5041e70255082/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:07:43.522343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ddb4c-93fb-4b78-b485-ce93318f309d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:43.523562 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ddb4c-93fb-4b78-b485-ce93318f309d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:43.524385 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ddb4c-93fb-4b78-b485-ce93318f309d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:43.567642 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8b4ddb4c-93fb-4b78-b485-ce93318f309d None None] MFA Rules not processed for user `38f31b382dea44dc9ae5041e70255082`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:07:43.569003 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8b4ddb4c-93fb-4b78-b485-ce93318f309d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:07:43.661224 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1415/2831] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:07:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:07:43.673288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57ed4d5-c7c5-4e69-85b5-b67744ee505d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:43.675372 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f57ed4d5-c7c5-4e69-85b5-b67744ee505d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:07:43.770816 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57ed4d5-c7c5-4e69-85b5-b67744ee505d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:43.779287 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb92301-81a4-4a49-a724-2d4142bf6c5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:43.780543 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb92301-81a4-4a49-a724-2d4142bf6c5e service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:43.780838 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb92301-81a4-4a49-a724-2d4142bf6c5e service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:43.781125 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb92301-81a4-4a49-a724-2d4142bf6c5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:43.792015 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb92301-81a4-4a49-a724-2d4142bf6c5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:43.793587 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb92301-81a4-4a49-a724-2d4142bf6c5e service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:43.806393 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1417/2832] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:07:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:07:43.828229 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ddb4c-93fb-4b78-b485-ce93318f309d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:43.829659 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8b4ddb4c-93fb-4b78-b485-ce93318f309d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:07:43.911980 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ddb4c-93fb-4b78-b485-ce93318f309d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:43.917941 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab2c05f-26a7-4f6e-b0cc-169bcacf79ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:43.918830 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2c05f-26a7-4f6e-b0cc-169bcacf79ba service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:43.919256 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2c05f-26a7-4f6e-b0cc-169bcacf79ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:43.919454 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2c05f-26a7-4f6e-b0cc-169bcacf79ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:43.921412 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0ab2c05f-26a7-4f6e-b0cc-169bcacf79ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:07:43.937334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab2c05f-26a7-4f6e-b0cc-169bcacf79ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:43.938794 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab2c05f-26a7-4f6e-b0cc-169bcacf79ba service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:43.943802 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:07:43.953604 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1416/2833] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:07:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:07:44.302627 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb92301-81a4-4a49-a724-2d4142bf6c5e service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:44.314494 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb92301-81a4-4a49-a724-2d4142bf6c5e service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:44.318506 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af3207a-4e3d-41d1-a7a2-e3083dee06e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:44.319179 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3207a-4e3d-41d1-a7a2-e3083dee06e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:44.319549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3207a-4e3d-41d1-a7a2-e3083dee06e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:44.320416 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3207a-4e3d-41d1-a7a2-e3083dee06e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:44.322251 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5af3207a-4e3d-41d1-a7a2-e3083dee06e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:07:44.337695 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af3207a-4e3d-41d1-a7a2-e3083dee06e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:44.339159 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af3207a-4e3d-41d1-a7a2-e3083dee06e7 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:44.352367 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1418/2834] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:07:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:07:53.954052 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:07:54.346470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=71,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:07:54.529079 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab2c05f-26a7-4f6e-b0cc-169bcacf79ba service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:54.592568 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab2c05f-26a7-4f6e-b0cc-169bcacf79ba service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:54.597347 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6cbfb-845f-4c95-99b6-7662ff8d5fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:54.598230 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6cbfb-845f-4c95-99b6-7662ff8d5fca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:54.598516 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6cbfb-845f-4c95-99b6-7662ff8d5fca admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:54.598795 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6cbfb-845f-4c95-99b6-7662ff8d5fca admin admin] PATH_INFO: `/v3/users/38f31b382dea44dc9ae5041e70255082` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:54.600063 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d6cbfb-845f-4c95-99b6-7662ff8d5fca admin admin] RBAC: Authorizing `identity:delete_user(user_id=38f31b382dea44dc9ae5041e70255082)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:54.602037 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d6cbfb-845f-4c95-99b6-7662ff8d5fca admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:54.669921 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d7d6cbfb-845f-4c95-99b6-7662ff8d5fca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38f31b382dea44dc9ae5041e70255082'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:07:54.680630 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d7d6cbfb-845f-4c95-99b6-7662ff8d5fca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38f31b382dea44dc9ae5041e70255082'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:07:54.690170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d7d6cbfb-845f-4c95-99b6-7662ff8d5fca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38f31b382dea44dc9ae5041e70255082'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:07:54.690844 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d7d6cbfb-845f-4c95-99b6-7662ff8d5fca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38f31b382dea44dc9ae5041e70255082'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:07:54.691469 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d7d6cbfb-845f-4c95-99b6-7662ff8d5fca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38f31b382dea44dc9ae5041e70255082'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:07:54.710755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1417/2835] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:07:54 2022] DELETE /identity/v3/users/38f31b382dea44dc9ae5041e70255082 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:07:54.716663 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ca8e7-1138-4d77-8911-6cccd4c84fbe None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:54.716957 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ca8e7-1138-4d77-8911-6cccd4c84fbe None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:54.717240 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ca8e7-1138-4d77-8911-6cccd4c84fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:54.738779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f48ca8e7-1138-4d77-8911-6cccd4c84fbe None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:07:54.740067 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f48ca8e7-1138-4d77-8911-6cccd4c84fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:07:54.820750 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1419/2836] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:07:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:07:54.832592 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6cbfb-845f-4c95-99b6-7662ff8d5fca admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:54.834328 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d7d6cbfb-845f-4c95-99b6-7662ff8d5fca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:07:54.905891 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6cbfb-845f-4c95-99b6-7662ff8d5fca admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:54.911736 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b696ea4-acb4-4a6d-9f3e-32b72fc2d989 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:54.912442 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b696ea4-acb4-4a6d-9f3e-32b72fc2d989 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:54.912728 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b696ea4-acb4-4a6d-9f3e-32b72fc2d989 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:54.913011 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b696ea4-acb4-4a6d-9f3e-32b72fc2d989 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:54.933135 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b696ea4-acb4-4a6d-9f3e-32b72fc2d989 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:54.934605 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b696ea4-acb4-4a6d-9f3e-32b72fc2d989 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:54.945195 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1418/2837] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:07:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:07:55.245772 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ca8e7-1138-4d77-8911-6cccd4c84fbe None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:55.247308 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f48ca8e7-1138-4d77-8911-6cccd4c84fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:07:55.263344 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ca8e7-1138-4d77-8911-6cccd4c84fbe None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:55.267452 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef97afd-65cf-4a50-a75d-0c7d80cf8dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:55.269813 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef97afd-65cf-4a50-a75d-0c7d80cf8dc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:55.269813 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef97afd-65cf-4a50-a75d-0c7d80cf8dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:55.270037 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef97afd-65cf-4a50-a75d-0c7d80cf8dc0 admin admin] PATH_INFO: `/v3/projects/d5748d52c41f4de1b9913d3cb1ffd134` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:55.271331 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef97afd-65cf-4a50-a75d-0c7d80cf8dc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5748d52c41f4de1b9913d3cb1ffd134)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:55.274023 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef97afd-65cf-4a50-a75d-0c7d80cf8dc0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:55.338249 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-aef97afd-65cf-4a50-a75d-0c7d80cf8dc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5748d52c41f4de1b9913d3cb1ffd134', 'request_id': 'req-aef97afd-65cf-4a50-a75d-0c7d80cf8dc0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:07:55.346029 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-aef97afd-65cf-4a50-a75d-0c7d80cf8dc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5748d52c41f4de1b9913d3cb1ffd134', 'request_id': 'req-aef97afd-65cf-4a50-a75d-0c7d80cf8dc0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:07:55.351789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-aef97afd-65cf-4a50-a75d-0c7d80cf8dc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5748d52c41f4de1b9913d3cb1ffd134', 'request_id': 'req-aef97afd-65cf-4a50-a75d-0c7d80cf8dc0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:07:55.364390 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-aef97afd-65cf-4a50-a75d-0c7d80cf8dc0 admin admin] The token cache is being invalidate because project d5748d52c41f4de1b9913d3cb1ffd134 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:07:55.364604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-aef97afd-65cf-4a50-a75d-0c7d80cf8dc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:07:55.365267 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-aef97afd-65cf-4a50-a75d-0c7d80cf8dc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:07:55.366799 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1420/2838] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:07:55 2022] DELETE /identity/v3/projects/d5748d52c41f4de1b9913d3cb1ffd134 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 20:07:55.373438 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a5d6b-eee5-4815-a816-cc080302eff4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:55.373959 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a5d6b-eee5-4815-a816-cc080302eff4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:55.374075 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a5d6b-eee5-4815-a816-cc080302eff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:55.395503 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-735a5d6b-eee5-4815-a816-cc080302eff4 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:07:55.396781 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-735a5d6b-eee5-4815-a816-cc080302eff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:07:55.416901 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:07:55.460609 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1419/2839] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:07:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:07:55.471251 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3d409-ea76-4050-8599-50fc4d78b188 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:55.471612 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3d409-ea76-4050-8599-50fc4d78b188 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:55.471888 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3d409-ea76-4050-8599-50fc4d78b188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:55.493295 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ecb3d409-ea76-4050-8599-50fc4d78b188 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:07:55.495240 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ecb3d409-ea76-4050-8599-50fc4d78b188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:07:55.503366 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1421/2840] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:07:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:07:55.508050 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a5d6b-eee5-4815-a816-cc080302eff4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:55.518313 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a5d6b-eee5-4815-a816-cc080302eff4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:55.523210 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9fcde7-e7b2-45f9-bc4a-757f67be163f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:55.523910 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9fcde7-e7b2-45f9-bc4a-757f67be163f admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:55.524192 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9fcde7-e7b2-45f9-bc4a-757f67be163f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:55.524471 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9fcde7-e7b2-45f9-bc4a-757f67be163f admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:55.525808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9fcde7-e7b2-45f9-bc4a-757f67be163f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 20:07:55.526120 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9fcde7-e7b2-45f9-bc4a-757f67be163f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:55.527761 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9fcde7-e7b2-45f9-bc4a-757f67be163f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:55.541843 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1420/2841] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 20:07:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 20:07:55.547265 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3d409-ea76-4050-8599-50fc4d78b188 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:55.557419 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3d409-ea76-4050-8599-50fc4d78b188 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:55.562143 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7083412d-2222-40f0-882a-b84a44d12013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:55.562817 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083412d-2222-40f0-882a-b84a44d12013 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:55.563097 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083412d-2222-40f0-882a-b84a44d12013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:55.563373 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083412d-2222-40f0-882a-b84a44d12013 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:55.564116 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7083412d-2222-40f0-882a-b84a44d12013 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:55.566032 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7083412d-2222-40f0-882a-b84a44d12013 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:55.597442 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1422/2842] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:07:55 2022] POST /identity/v3/projects => generated 401 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:07:55.602091 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9fcde7-e7b2-45f9-bc4a-757f67be163f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:55.613792 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9fcde7-e7b2-45f9-bc4a-757f67be163f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:55.674744 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3586ca-21ac-4850-8c97-15e5e629f960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:55.675423 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3586ca-21ac-4850-8c97-15e5e629f960 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:55.675743 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3586ca-21ac-4850-8c97-15e5e629f960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:55.676024 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3586ca-21ac-4850-8c97-15e5e629f960 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:55.676752 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3586ca-21ac-4850-8c97-15e5e629f960 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:55.678688 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3586ca-21ac-4850-8c97-15e5e629f960 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:55.706604 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1421/2843] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:07:55 2022] POST /identity/v3/users => generated 301 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:07:55.708110 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7083412d-2222-40f0-882a-b84a44d12013 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:55.720036 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7083412d-2222-40f0-882a-b84a44d12013 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:55.725004 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb2781-0fe3-4a49-80f9-eec51ca642ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:55.725790 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb2781-0fe3-4a49-80f9-eec51ca642ac admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:55.726138 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb2781-0fe3-4a49-80f9-eec51ca642ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:55.726481 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb2781-0fe3-4a49-80f9-eec51ca642ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:55.727285 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb2781-0fe3-4a49-80f9-eec51ca642ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:55.729777 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb2781-0fe3-4a49-80f9-eec51ca642ac admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:55.741718 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1423/2844] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:07:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:07:55.746792 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3586ca-21ac-4850-8c97-15e5e629f960 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:55.756993 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3586ca-21ac-4850-8c97-15e5e629f960 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:55.762295 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b903dd5-985e-47c6-90b7-f5e7e6df35a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:55.762969 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b903dd5-985e-47c6-90b7-f5e7e6df35a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:55.763253 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b903dd5-985e-47c6-90b7-f5e7e6df35a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:55.763601 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b903dd5-985e-47c6-90b7-f5e7e6df35a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:55.764289 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b903dd5-985e-47c6-90b7-f5e7e6df35a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:55.765901 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b903dd5-985e-47c6-90b7-f5e7e6df35a1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:55.780965 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1422/2845] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:07:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:07:55.782400 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb2781-0fe3-4a49-80f9-eec51ca642ac admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:55.793373 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb2781-0fe3-4a49-80f9-eec51ca642ac admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:55.798167 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994bcc23-a3d8-4b6d-8cad-03facb74caf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:55.799051 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994bcc23-a3d8-4b6d-8cad-03facb74caf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:55.799362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994bcc23-a3d8-4b6d-8cad-03facb74caf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:55.799672 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994bcc23-a3d8-4b6d-8cad-03facb74caf1 admin admin] PATH_INFO: `/v3/projects/f34086ef489d49f5b86e230fe1828aca/users/6e2ffe809eda4674a2d88b12f3b95744/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:55.818141 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994bcc23-a3d8-4b6d-8cad-03facb74caf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f34086ef489d49f5b86e230fe1828aca, user_id=6e2ffe809eda4674a2d88b12f3b95744, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:55.821998 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994bcc23-a3d8-4b6d-8cad-03facb74caf1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:55.835484 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1424/2846] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:07:55 2022] PUT /identity/v3/projects/f34086ef489d49f5b86e230fe1828aca/users/6e2ffe809eda4674a2d88b12f3b95744/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:07:55.849015 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00422f-6740-467d-9641-9ef1b654b840 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:55.849309 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00422f-6740-467d-9641-9ef1b654b840 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:55.849590 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00422f-6740-467d-9641-9ef1b654b840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:55.880059 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6e00422f-6740-467d-9641-9ef1b654b840 None None] MFA Rules not processed for user `6e2ffe809eda4674a2d88b12f3b95744`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:07:55.881291 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6e00422f-6740-467d-9641-9ef1b654b840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:07:55.951359 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1423/2847] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:07:55 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 20:07:55.956355 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994bcc23-a3d8-4b6d-8cad-03facb74caf1 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:55.978167 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:07:56.028414 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994bcc23-a3d8-4b6d-8cad-03facb74caf1 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:56.034392 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa47e14-1fbe-421f-810e-f33e78ca571c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:56.035080 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa47e14-1fbe-421f-810e-f33e78ca571c admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:56.035366 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa47e14-1fbe-421f-810e-f33e78ca571c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:56.035691 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa47e14-1fbe-421f-810e-f33e78ca571c admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:56.036443 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa47e14-1fbe-421f-810e-f33e78ca571c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:56.038597 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa47e14-1fbe-421f-810e-f33e78ca571c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:56.068233 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1425/2848] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:07:55 2022] POST /identity/v3/projects => generated 399 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:07:56.073356 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00422f-6740-467d-9641-9ef1b654b840 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:56.084321 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00422f-6740-467d-9641-9ef1b654b840 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:56.098261 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:07:56.146576 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc074d3-d963-4447-8e7a-797667efcab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:56.147261 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc074d3-d963-4447-8e7a-797667efcab2 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:56.147579 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc074d3-d963-4447-8e7a-797667efcab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:56.147891 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc074d3-d963-4447-8e7a-797667efcab2 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:56.148605 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc074d3-d963-4447-8e7a-797667efcab2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:56.151016 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc074d3-d963-4447-8e7a-797667efcab2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:56.174863 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1424/2849] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:07:56 2022] POST /identity/v3/users => generated 306 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:07:56.178878 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa47e14-1fbe-421f-810e-f33e78ca571c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:56.190549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa47e14-1fbe-421f-810e-f33e78ca571c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:56.195549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959cd1aa-1f65-48c9-8222-568116c07113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:56.196262 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959cd1aa-1f65-48c9-8222-568116c07113 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:56.197195 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959cd1aa-1f65-48c9-8222-568116c07113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:56.197439 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959cd1aa-1f65-48c9-8222-568116c07113 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:56.198226 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959cd1aa-1f65-48c9-8222-568116c07113 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:56.199825 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959cd1aa-1f65-48c9-8222-568116c07113 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:56.211283 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1426/2850] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:07:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:07:56.216459 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc074d3-d963-4447-8e7a-797667efcab2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:56.226569 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc074d3-d963-4447-8e7a-797667efcab2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:56.231416 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fec488-de65-4933-a05c-10bdb0fcbd51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:56.232326 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fec488-de65-4933-a05c-10bdb0fcbd51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:56.232618 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fec488-de65-4933-a05c-10bdb0fcbd51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:56.232897 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fec488-de65-4933-a05c-10bdb0fcbd51 admin admin] PATH_INFO: `/v3/projects/8186dbb8290e47acbf5edd6f0baf76b4/users/f3e5a4d745e6442eb3e266d2f4c7a6fd/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:56.248998 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fec488-de65-4933-a05c-10bdb0fcbd51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8186dbb8290e47acbf5edd6f0baf76b4, user_id=f3e5a4d745e6442eb3e266d2f4c7a6fd, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:56.251722 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fec488-de65-4933-a05c-10bdb0fcbd51 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:56.267293 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1425/2851] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:07:56 2022] PUT /identity/v3/projects/8186dbb8290e47acbf5edd6f0baf76b4/users/f3e5a4d745e6442eb3e266d2f4c7a6fd/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:07:56.271397 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959cd1aa-1f65-48c9-8222-568116c07113 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:56.282762 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959cd1aa-1f65-48c9-8222-568116c07113 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:56.345041 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0738b8ef-9bfe-432d-814b-69c9b65f51f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:56.345713 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738b8ef-9bfe-432d-814b-69c9b65f51f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:56.345998 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738b8ef-9bfe-432d-814b-69c9b65f51f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:56.346273 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738b8ef-9bfe-432d-814b-69c9b65f51f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:56.347306 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0738b8ef-9bfe-432d-814b-69c9b65f51f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:56.349074 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0738b8ef-9bfe-432d-814b-69c9b65f51f5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:56.359629 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1427/2852] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:07:56 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 20:07:56.363882 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fec488-de65-4933-a05c-10bdb0fcbd51 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:56.375546 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fec488-de65-4933-a05c-10bdb0fcbd51 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:56.380861 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d1e75-812a-466b-8638-76ee09086158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:56.381635 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d1e75-812a-466b-8638-76ee09086158 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:56.381914 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d1e75-812a-466b-8638-76ee09086158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:56.382186 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d1e75-812a-466b-8638-76ee09086158 admin admin] PATH_INFO: `/v3/projects/8186dbb8290e47acbf5edd6f0baf76b4/users/f3e5a4d745e6442eb3e266d2f4c7a6fd/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:56.385801 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d1e75-812a-466b-8638-76ee09086158 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8186dbb8290e47acbf5edd6f0baf76b4, user_id=f3e5a4d745e6442eb3e266d2f4c7a6fd, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:56.388568 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d1e75-812a-466b-8638-76ee09086158 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:56.399948 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1426/2853] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:07:56 2022] PUT /identity/v3/projects/8186dbb8290e47acbf5edd6f0baf76b4/users/f3e5a4d745e6442eb3e266d2f4c7a6fd/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:07:56.410292 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c47bc7-00f5-443f-9552-e50ff1f8b1a9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:56.410579 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c47bc7-00f5-443f-9552-e50ff1f8b1a9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:56.410863 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c47bc7-00f5-443f-9552-e50ff1f8b1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:56.441969 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-64c47bc7-00f5-443f-9552-e50ff1f8b1a9 None None] MFA Rules not processed for user `f3e5a4d745e6442eb3e266d2f4c7a6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:07:56.443206 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-64c47bc7-00f5-443f-9552-e50ff1f8b1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:07:56.519449 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1428/2854] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:07:56 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 20:07:56.532600 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d1e75-812a-466b-8638-76ee09086158 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:56.534396 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d55d1e75-812a-466b-8638-76ee09086158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:07:56.600269 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d1e75-812a-466b-8638-76ee09086158 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:56.604363 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42755659-6c35-49aa-ba1d-f15fa344a74f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:56.605108 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42755659-6c35-49aa-ba1d-f15fa344a74f service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:56.605393 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42755659-6c35-49aa-ba1d-f15fa344a74f service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:56.605802 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42755659-6c35-49aa-ba1d-f15fa344a74f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:56.616917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42755659-6c35-49aa-ba1d-f15fa344a74f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:56.618270 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42755659-6c35-49aa-ba1d-f15fa344a74f service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:56.629014 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1427/2855] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:07:56 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:07:56.713890 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c47bc7-00f5-443f-9552-e50ff1f8b1a9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:07:56.724329 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c47bc7-00f5-443f-9552-e50ff1f8b1a9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:07:56.728167 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cb40d8-c6dc-45c0-b413-ec2623e92153 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:07:56.729008 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb40d8-c6dc-45c0-b413-ec2623e92153 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:07:56.729390 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb40d8-c6dc-45c0-b413-ec2623e92153 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:07:56.729738 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb40d8-c6dc-45c0-b413-ec2623e92153 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:07:56.744437 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:07:56.794616 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cb40d8-c6dc-45c0-b413-ec2623e92153 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:07:56.796024 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cb40d8-c6dc-45c0-b413-ec2623e92153 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:07:56.805605 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1429/2856] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:07:56 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:08:00.518601 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23442ecd-26b9-48b7-810c-39574e054380 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:00.519185 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23442ecd-26b9-48b7-810c-39574e054380 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:00.519185 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23442ecd-26b9-48b7-810c-39574e054380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:00.540380 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-23442ecd-26b9-48b7-810c-39574e054380 None None] MFA Rules not processed for user `8143c33eabd74b1b9edc84e8ff155903`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:08:00.541857 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-23442ecd-26b9-48b7-810c-39574e054380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:00.602274 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1428/2857] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:08:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:08:00.916688 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cb40d8-c6dc-45c0-b413-ec2623e92153 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:00.942759 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23442ecd-26b9-48b7-810c-39574e054380 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:00.944225 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-23442ecd-26b9-48b7-810c-39574e054380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:00.966898 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:08:00.998348 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cb40d8-c6dc-45c0-b413-ec2623e92153 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:01.004721 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365316ef-fa2e-4ce1-933d-810d3c27f843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:01.005645 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365316ef-fa2e-4ce1-933d-810d3c27f843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:01.005954 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365316ef-fa2e-4ce1-933d-810d3c27f843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:01.006235 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365316ef-fa2e-4ce1-933d-810d3c27f843 admin admin] PATH_INFO: `/v3/users/6e2ffe809eda4674a2d88b12f3b95744` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:01.007539 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365316ef-fa2e-4ce1-933d-810d3c27f843 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e2ffe809eda4674a2d88b12f3b95744)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:01.010038 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365316ef-fa2e-4ce1-933d-810d3c27f843 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:01.012028 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23442ecd-26b9-48b7-810c-39574e054380 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:01.018361 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c02e2f-1a6f-4879-bce7-9799f2f97d54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:01.019078 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c02e2f-1a6f-4879-bce7-9799f2f97d54 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:01.019373 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c02e2f-1a6f-4879-bce7-9799f2f97d54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:01.019707 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c02e2f-1a6f-4879-bce7-9799f2f97d54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:01.021654 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a1c02e2f-1a6f-4879-bce7-9799f2f97d54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:01.033067 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c02e2f-1a6f-4879-bce7-9799f2f97d54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:01.034670 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c02e2f-1a6f-4879-bce7-9799f2f97d54 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:01.044954 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1429/2858] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:08:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 20:08:01.071202 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-365316ef-fa2e-4ce1-933d-810d3c27f843 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e2ffe809eda4674a2d88b12f3b95744'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:01.081312 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-365316ef-fa2e-4ce1-933d-810d3c27f843 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e2ffe809eda4674a2d88b12f3b95744'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:01.090148 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-365316ef-fa2e-4ce1-933d-810d3c27f843 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e2ffe809eda4674a2d88b12f3b95744'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:01.090893 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-365316ef-fa2e-4ce1-933d-810d3c27f843 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e2ffe809eda4674a2d88b12f3b95744'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:01.104577 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-365316ef-fa2e-4ce1-933d-810d3c27f843 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e2ffe809eda4674a2d88b12f3b95744'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:01.110983 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1430/2859] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:08:00 2022] DELETE /identity/v3/users/6e2ffe809eda4674a2d88b12f3b95744 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:08:01.116041 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2012c07-d3c5-4759-b0cd-610f66f942cb None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:01.116349 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2012c07-d3c5-4759-b0cd-610f66f942cb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:01.116627 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2012c07-d3c5-4759-b0cd-610f66f942cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:01.138161 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c2012c07-d3c5-4759-b0cd-610f66f942cb None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:08:01.139431 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c2012c07-d3c5-4759-b0cd-610f66f942cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:01.223751 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1430/2860] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:08:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:08:01.236763 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365316ef-fa2e-4ce1-933d-810d3c27f843 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:01.238197 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-365316ef-fa2e-4ce1-933d-810d3c27f843 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:01.282216 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:08:01.308036 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365316ef-fa2e-4ce1-933d-810d3c27f843 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:01.312022 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0ec1d-e446-47fd-a364-db56b88f2bea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:01.312675 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0ec1d-e446-47fd-a364-db56b88f2bea service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:01.312962 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0ec1d-e446-47fd-a364-db56b88f2bea service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:01.313243 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0ec1d-e446-47fd-a364-db56b88f2bea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:01.324045 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc0ec1d-e446-47fd-a364-db56b88f2bea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:01.325498 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc0ec1d-e446-47fd-a364-db56b88f2bea service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:01.335630 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1431/2861] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:08:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:08:01.537509 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2012c07-d3c5-4759-b0cd-610f66f942cb None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:01.539708 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c2012c07-d3c5-4759-b0cd-610f66f942cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:01.634446 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2012c07-d3c5-4759-b0cd-610f66f942cb None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:01.639689 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37adef08-4a38-4160-9fe9-ad44a72b6124 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0422d6695bf14925bf0f993cb16e64ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:01.640369 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37adef08-4a38-4160-9fe9-ad44a72b6124 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:01.640651 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37adef08-4a38-4160-9fe9-ad44a72b6124 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:01.641548 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37adef08-4a38-4160-9fe9-ad44a72b6124 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:01.643350 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-37adef08-4a38-4160-9fe9-ad44a72b6124 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:01.658086 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37adef08-4a38-4160-9fe9-ad44a72b6124 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:01.662484 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37adef08-4a38-4160-9fe9-ad44a72b6124 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:01.663761 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0ec1d-e446-47fd-a364-db56b88f2bea service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:01.665991 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fdc0ec1d-e446-47fd-a364-db56b88f2bea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:01.671918 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1431/2862] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:08:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:08:01.682636 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0ec1d-e446-47fd-a364-db56b88f2bea service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:01.687427 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8446353b-4200-4e6a-8477-d26a4c69562f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:01.688316 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446353b-4200-4e6a-8477-d26a4c69562f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:01.688643 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446353b-4200-4e6a-8477-d26a4c69562f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:01.688937 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446353b-4200-4e6a-8477-d26a4c69562f admin admin] PATH_INFO: `/v3/projects/f34086ef489d49f5b86e230fe1828aca` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:01.690147 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8446353b-4200-4e6a-8477-d26a4c69562f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f34086ef489d49f5b86e230fe1828aca)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:01.692703 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8446353b-4200-4e6a-8477-d26a4c69562f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:01.763220 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8446353b-4200-4e6a-8477-d26a4c69562f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f34086ef489d49f5b86e230fe1828aca', 'request_id': 'req-8446353b-4200-4e6a-8477-d26a4c69562f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:01.774297 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8446353b-4200-4e6a-8477-d26a4c69562f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f34086ef489d49f5b86e230fe1828aca', 'request_id': 'req-8446353b-4200-4e6a-8477-d26a4c69562f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:01.782419 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8446353b-4200-4e6a-8477-d26a4c69562f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f34086ef489d49f5b86e230fe1828aca', 'request_id': 'req-8446353b-4200-4e6a-8477-d26a4c69562f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:01.794556 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8446353b-4200-4e6a-8477-d26a4c69562f admin admin] The token cache is being invalidate because project f34086ef489d49f5b86e230fe1828aca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:08:01.794845 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8446353b-4200-4e6a-8477-d26a4c69562f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:01.795810 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8446353b-4200-4e6a-8477-d26a4c69562f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:01.797380 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1432/2863] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:08:01 2022] DELETE /identity/v3/projects/f34086ef489d49f5b86e230fe1828aca => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 20:08:01.802426 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37adef08-4a38-4160-9fe9-ad44a72b6124 service placement] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:01.814950 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37adef08-4a38-4160-9fe9-ad44a72b6124 service placement] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:01.880648 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:08:01.899194 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00470135-d70f-4c74-9513-7cec763147d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:01.899747 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00470135-d70f-4c74-9513-7cec763147d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:01.900196 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00470135-d70f-4c74-9513-7cec763147d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:01.900706 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00470135-d70f-4c74-9513-7cec763147d3 admin admin] PATH_INFO: `/v3/users/f3e5a4d745e6442eb3e266d2f4c7a6fd` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:01.901980 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00470135-d70f-4c74-9513-7cec763147d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3e5a4d745e6442eb3e266d2f4c7a6fd)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:01.904021 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00470135-d70f-4c74-9513-7cec763147d3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:01.965424 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-00470135-d70f-4c74-9513-7cec763147d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3e5a4d745e6442eb3e266d2f4c7a6fd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:01.977014 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-00470135-d70f-4c74-9513-7cec763147d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3e5a4d745e6442eb3e266d2f4c7a6fd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:01.986759 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-00470135-d70f-4c74-9513-7cec763147d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3e5a4d745e6442eb3e266d2f4c7a6fd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:01.987601 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-00470135-d70f-4c74-9513-7cec763147d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3e5a4d745e6442eb3e266d2f4c7a6fd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:01.988353 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-00470135-d70f-4c74-9513-7cec763147d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3e5a4d745e6442eb3e266d2f4c7a6fd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:02.010593 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1432/2864] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:08:01 2022] DELETE /identity/v3/users/f3e5a4d745e6442eb3e266d2f4c7a6fd => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:08:02.331340 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8446353b-4200-4e6a-8477-d26a4c69562f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:02.333039 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8446353b-4200-4e6a-8477-d26a4c69562f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:02.348140 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8446353b-4200-4e6a-8477-d26a4c69562f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:02.425504 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea058b3f-3240-4b44-9dbc-83c6c6c22e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:02.426419 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea058b3f-3240-4b44-9dbc-83c6c6c22e1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:02.426775 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea058b3f-3240-4b44-9dbc-83c6c6c22e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:02.427052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea058b3f-3240-4b44-9dbc-83c6c6c22e1a admin admin] PATH_INFO: `/v3/projects/8186dbb8290e47acbf5edd6f0baf76b4` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:02.428563 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea058b3f-3240-4b44-9dbc-83c6c6c22e1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8186dbb8290e47acbf5edd6f0baf76b4)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:02.430497 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea058b3f-3240-4b44-9dbc-83c6c6c22e1a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:02.492968 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ea058b3f-3240-4b44-9dbc-83c6c6c22e1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8186dbb8290e47acbf5edd6f0baf76b4', 'request_id': 'req-ea058b3f-3240-4b44-9dbc-83c6c6c22e1a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:02.501373 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ea058b3f-3240-4b44-9dbc-83c6c6c22e1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8186dbb8290e47acbf5edd6f0baf76b4', 'request_id': 'req-ea058b3f-3240-4b44-9dbc-83c6c6c22e1a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:02.505703 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:08:02.508009 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ea058b3f-3240-4b44-9dbc-83c6c6c22e1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8186dbb8290e47acbf5edd6f0baf76b4', 'request_id': 'req-ea058b3f-3240-4b44-9dbc-83c6c6c22e1a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:02.521874 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ea058b3f-3240-4b44-9dbc-83c6c6c22e1a admin admin] The token cache is being invalidate because project 8186dbb8290e47acbf5edd6f0baf76b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:08:02.522165 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ea058b3f-3240-4b44-9dbc-83c6c6c22e1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:02.522815 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ea058b3f-3240-4b44-9dbc-83c6c6c22e1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:02.524379 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1433/2865] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:08:02 2022] DELETE /identity/v3/projects/8186dbb8290e47acbf5edd6f0baf76b4 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:08:02.530695 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6dc9b2-9935-44c3-be87-3f418f6945f4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:02.530989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6dc9b2-9935-44c3-be87-3f418f6945f4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:02.531273 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6dc9b2-9935-44c3-be87-3f418f6945f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:02.551778 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ba6dc9b2-9935-44c3-be87-3f418f6945f4 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:08:02.553165 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ba6dc9b2-9935-44c3-be87-3f418f6945f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:02.616417 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1433/2866] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:08:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:08:02.625187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f9fc2-2953-40bb-b7ec-d935eb7f81aa None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:02.625533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f9fc2-2953-40bb-b7ec-d935eb7f81aa None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:02.625764 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f9fc2-2953-40bb-b7ec-d935eb7f81aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:02.647396 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-374f9fc2-2953-40bb-b7ec-d935eb7f81aa None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:08:02.647983 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-374f9fc2-2953-40bb-b7ec-d935eb7f81aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:02.656392 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1434/2867] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:08:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:08:02.660635 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6dc9b2-9935-44c3-be87-3f418f6945f4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:02.672129 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6dc9b2-9935-44c3-be87-3f418f6945f4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:02.677158 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27dc00e-fcd8-4695-ba61-259a7179ea16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:02.677837 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27dc00e-fcd8-4695-ba61-259a7179ea16 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:02.678127 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27dc00e-fcd8-4695-ba61-259a7179ea16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:02.678401 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27dc00e-fcd8-4695-ba61-259a7179ea16 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:02.679238 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27dc00e-fcd8-4695-ba61-259a7179ea16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 20:08:02.679576 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27dc00e-fcd8-4695-ba61-259a7179ea16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:02.682337 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27dc00e-fcd8-4695-ba61-259a7179ea16 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:02.695086 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1434/2868] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 20:08:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 20:08:02.699754 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f9fc2-2953-40bb-b7ec-d935eb7f81aa None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:02.709725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f9fc2-2953-40bb-b7ec-d935eb7f81aa None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:02.715161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb3fbf0-f8a6-481c-a856-6c25af550508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:02.715865 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3fbf0-f8a6-481c-a856-6c25af550508 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:02.716155 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3fbf0-f8a6-481c-a856-6c25af550508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:02.716461 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3fbf0-f8a6-481c-a856-6c25af550508 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:02.717157 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb3fbf0-f8a6-481c-a856-6c25af550508 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:02.719282 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb3fbf0-f8a6-481c-a856-6c25af550508 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:02.756206 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1435/2869] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:08:02 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:08:02.757134 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27dc00e-fcd8-4695-ba61-259a7179ea16 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:02.768295 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27dc00e-fcd8-4695-ba61-259a7179ea16 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:02.831692 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9de30c-8684-418f-816a-f678d80e6143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:02.832385 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9de30c-8684-418f-816a-f678d80e6143 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:02.832681 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9de30c-8684-418f-816a-f678d80e6143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:02.832963 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9de30c-8684-418f-816a-f678d80e6143 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:02.833676 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9de30c-8684-418f-816a-f678d80e6143 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:02.835648 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9de30c-8684-418f-816a-f678d80e6143 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:02.838374 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:08:02.866194 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1435/2870] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:08:02 2022] POST /identity/v3/users => generated 289 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 20:08:02.868991 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb3fbf0-f8a6-481c-a856-6c25af550508 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:02.883608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb3fbf0-f8a6-481c-a856-6c25af550508 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:02.888922 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe007d0-3307-4171-9dd3-9ef41034080a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:02.889594 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe007d0-3307-4171-9dd3-9ef41034080a admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:02.889880 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe007d0-3307-4171-9dd3-9ef41034080a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:02.890159 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe007d0-3307-4171-9dd3-9ef41034080a admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:02.890904 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe007d0-3307-4171-9dd3-9ef41034080a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:02.892733 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe007d0-3307-4171-9dd3-9ef41034080a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:02.910751 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1436/2871] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:08:02.912107 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9de30c-8684-418f-816a-f678d80e6143 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:02.932038 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9de30c-8684-418f-816a-f678d80e6143 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:02.939015 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54754872-4b05-4b4f-b39c-b67d7180f511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:02.939767 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54754872-4b05-4b4f-b39c-b67d7180f511 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:02.940106 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54754872-4b05-4b4f-b39c-b67d7180f511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:02.940387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54754872-4b05-4b4f-b39c-b67d7180f511 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:02.941124 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54754872-4b05-4b4f-b39c-b67d7180f511 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:02.942955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54754872-4b05-4b4f-b39c-b67d7180f511 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:02.958834 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe007d0-3307-4171-9dd3-9ef41034080a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:02.959280 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1436/2872] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 20:08:02.970800 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe007d0-3307-4171-9dd3-9ef41034080a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:02.976270 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f6d219-abd6-48d5-92db-2a81ca82fa58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:02.977167 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f6d219-abd6-48d5-92db-2a81ca82fa58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:02.977452 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f6d219-abd6-48d5-92db-2a81ca82fa58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:02.979009 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f6d219-abd6-48d5-92db-2a81ca82fa58 admin admin] PATH_INFO: `/v3/projects/a6dca77e8b57479084b453d788f3d90b/users/9d346cc28ebb459da396876ffcfc265d/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:02.995253 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f6d219-abd6-48d5-92db-2a81ca82fa58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6dca77e8b57479084b453d788f3d90b, user_id=9d346cc28ebb459da396876ffcfc265d, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:02.997941 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f6d219-abd6-48d5-92db-2a81ca82fa58 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:03.012932 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1437/2873] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:08:02 2022] PUT /identity/v3/projects/a6dca77e8b57479084b453d788f3d90b/users/9d346cc28ebb459da396876ffcfc265d/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 20:08:03.023098 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ac87f-37cb-42f1-95ad-85bf347f13a9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:03.023378 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ac87f-37cb-42f1-95ad-85bf347f13a9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:03.023701 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ac87f-37cb-42f1-95ad-85bf347f13a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:03.053293 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3b0ac87f-37cb-42f1-95ad-85bf347f13a9 None None] MFA Rules not processed for user `9d346cc28ebb459da396876ffcfc265d`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:08:03.054657 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3b0ac87f-37cb-42f1-95ad-85bf347f13a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:03.157418 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1437/2874] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:08:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:08:03.171297 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f6d219-abd6-48d5-92db-2a81ca82fa58 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:03.172847 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-47f6d219-abd6-48d5-92db-2a81ca82fa58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:03.241395 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f6d219-abd6-48d5-92db-2a81ca82fa58 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:03.246296 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd91f0-0381-401c-8dd5-1e98089cb4fe service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98560723b49940d38a6a5e9cf3f5d3b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:03.247177 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd91f0-0381-401c-8dd5-1e98089cb4fe service glance] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:03.247289 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd91f0-0381-401c-8dd5-1e98089cb4fe service glance] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:03.247570 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd91f0-0381-401c-8dd5-1e98089cb4fe service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:03.257614 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bd91f0-0381-401c-8dd5-1e98089cb4fe service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:03.259153 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bd91f0-0381-401c-8dd5-1e98089cb4fe service glance] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:03.269917 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1438/2875] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:08:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:08:03.800168 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ac87f-37cb-42f1-95ad-85bf347f13a9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:03.801584 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3b0ac87f-37cb-42f1-95ad-85bf347f13a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:03.928763 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ac87f-37cb-42f1-95ad-85bf347f13a9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:03.943081 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791b266f-3584-489c-8f8d-97910c96f6b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:03.943975 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b266f-3584-489c-8f8d-97910c96f6b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:03.944383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b266f-3584-489c-8f8d-97910c96f6b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:03.944745 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791b266f-3584-489c-8f8d-97910c96f6b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:03.948540 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-791b266f-3584-489c-8f8d-97910c96f6b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:03.961855 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791b266f-3584-489c-8f8d-97910c96f6b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:03.964854 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791b266f-3584-489c-8f8d-97910c96f6b9 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:03.974864 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1438/2876] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:08:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:08:04.692116 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd91f0-0381-401c-8dd5-1e98089cb4fe service glance] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:04.694457 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-68bd91f0-0381-401c-8dd5-1e98089cb4fe service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:04.810060 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd91f0-0381-401c-8dd5-1e98089cb4fe service glance] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:04.816045 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12e368-6f53-46f4-b7b4-70115c85671c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:04.816936 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12e368-6f53-46f4-b7b4-70115c85671c service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:04.817397 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12e368-6f53-46f4-b7b4-70115c85671c service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:04.817841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12e368-6f53-46f4-b7b4-70115c85671c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:04.820571 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-db12e368-6f53-46f4-b7b4-70115c85671c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:04.834524 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db12e368-6f53-46f4-b7b4-70115c85671c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:04.836203 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db12e368-6f53-46f4-b7b4-70115c85671c service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:04.856363 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1439/2877] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:08:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:08:13.969995 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=66,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:08:14.845649 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=1,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:08:16.629259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791b266f-3584-489c-8f8d-97910c96f6b9 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:16.630838 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-791b266f-3584-489c-8f8d-97910c96f6b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:16.649588 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791b266f-3584-489c-8f8d-97910c96f6b9 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:16.655327 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfd488a-ba24-47b9-94ed-3e819da3120d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:16.656066 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd488a-ba24-47b9-94ed-3e819da3120d service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:16.656353 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd488a-ba24-47b9-94ed-3e819da3120d service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:16.656640 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd488a-ba24-47b9-94ed-3e819da3120d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:16.658450 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cbfd488a-ba24-47b9-94ed-3e819da3120d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:16.672263 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfd488a-ba24-47b9-94ed-3e819da3120d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:16.673969 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfd488a-ba24-47b9-94ed-3e819da3120d service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:16.686944 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1439/2878] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:08:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:08:19.783175 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12e368-6f53-46f4-b7b4-70115c85671c service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:19.784652 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-db12e368-6f53-46f4-b7b4-70115c85671c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:19.796440 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12e368-6f53-46f4-b7b4-70115c85671c service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:19.800027 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c031c07a-5107-4e4b-94a8-d1c1f7254832 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:19.800682 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031c07a-5107-4e4b-94a8-d1c1f7254832 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:19.800970 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031c07a-5107-4e4b-94a8-d1c1f7254832 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:19.801247 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c031c07a-5107-4e4b-94a8-d1c1f7254832 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:19.803024 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c031c07a-5107-4e4b-94a8-d1c1f7254832 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:19.872807 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c031c07a-5107-4e4b-94a8-d1c1f7254832 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:19.874272 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c031c07a-5107-4e4b-94a8-d1c1f7254832 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:19.885125 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1440/2879] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:08:19 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:08:21.772998 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfd488a-ba24-47b9-94ed-3e819da3120d service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:21.784900 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfd488a-ba24-47b9-94ed-3e819da3120d service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:21.790520 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276bacc-c196-4343-86d5-64870275c7bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:21.791211 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276bacc-c196-4343-86d5-64870275c7bb service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:21.791464 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276bacc-c196-4343-86d5-64870275c7bb service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:21.791755 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276bacc-c196-4343-86d5-64870275c7bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:21.794171 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3276bacc-c196-4343-86d5-64870275c7bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:21.808176 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3276bacc-c196-4343-86d5-64870275c7bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:21.809802 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3276bacc-c196-4343-86d5-64870275c7bb service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:21.824090 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1440/2880] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:08:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:08:22.200611 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c031c07a-5107-4e4b-94a8-d1c1f7254832 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:22.308394 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c031c07a-5107-4e4b-94a8-d1c1f7254832 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:22.314676 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7429b18-78ae-40bc-872d-c90845af653a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:22.315612 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7429b18-78ae-40bc-872d-c90845af653a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:22.316085 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7429b18-78ae-40bc-872d-c90845af653a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:22.316583 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7429b18-78ae-40bc-872d-c90845af653a admin admin] PATH_INFO: `/v3/users/9d346cc28ebb459da396876ffcfc265d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:22.317974 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7429b18-78ae-40bc-872d-c90845af653a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d346cc28ebb459da396876ffcfc265d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:22.320417 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7429b18-78ae-40bc-872d-c90845af653a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:22.382798 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c7429b18-78ae-40bc-872d-c90845af653a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d346cc28ebb459da396876ffcfc265d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:22.393244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c7429b18-78ae-40bc-872d-c90845af653a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d346cc28ebb459da396876ffcfc265d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:22.402169 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c7429b18-78ae-40bc-872d-c90845af653a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d346cc28ebb459da396876ffcfc265d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:22.403390 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c7429b18-78ae-40bc-872d-c90845af653a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d346cc28ebb459da396876ffcfc265d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:22.417731 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:08:22.420281 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c7429b18-78ae-40bc-872d-c90845af653a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d346cc28ebb459da396876ffcfc265d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:22.430052 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1441/2881] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:08:22 2022] DELETE /identity/v3/users/9d346cc28ebb459da396876ffcfc265d => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:08:22.436052 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c91a0-a260-4e24-a64d-f081d52a54cd None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:22.436670 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c91a0-a260-4e24-a64d-f081d52a54cd None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:22.437201 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c91a0-a260-4e24-a64d-f081d52a54cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:22.478365 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3e5c91a0-a260-4e24-a64d-f081d52a54cd None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:08:22.480157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3e5c91a0-a260-4e24-a64d-f081d52a54cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:22.550583 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1441/2882] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:08:22.561162 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7429b18-78ae-40bc-872d-c90845af653a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:22.562715 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c7429b18-78ae-40bc-872d-c90845af653a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:22.626506 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7429b18-78ae-40bc-872d-c90845af653a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:22.630410 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3df15-f4c2-46e7-a2a3-f9dc00425516 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:22.631070 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3df15-f4c2-46e7-a2a3-f9dc00425516 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:22.631357 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3df15-f4c2-46e7-a2a3-f9dc00425516 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:22.631668 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3df15-f4c2-46e7-a2a3-f9dc00425516 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:22.644473 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d3df15-f4c2-46e7-a2a3-f9dc00425516 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:22.646258 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d3df15-f4c2-46e7-a2a3-f9dc00425516 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:22.658727 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1442/2883] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:08:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:08:22.969428 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c91a0-a260-4e24-a64d-f081d52a54cd None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:22.970920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3e5c91a0-a260-4e24-a64d-f081d52a54cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:22.987743 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c91a0-a260-4e24-a64d-f081d52a54cd None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:22.996415 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ff298-b41d-4f22-a02d-960d1594ff1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:22.997546 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ff298-b41d-4f22-a02d-960d1594ff1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:22.997903 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ff298-b41d-4f22-a02d-960d1594ff1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:22.998319 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ff298-b41d-4f22-a02d-960d1594ff1c admin admin] PATH_INFO: `/v3/projects/a6dca77e8b57479084b453d788f3d90b` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:22.999788 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3ff298-b41d-4f22-a02d-960d1594ff1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6dca77e8b57479084b453d788f3d90b)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:23.001818 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3ff298-b41d-4f22-a02d-960d1594ff1c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:23.063406 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-df3ff298-b41d-4f22-a02d-960d1594ff1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6dca77e8b57479084b453d788f3d90b', 'request_id': 'req-df3ff298-b41d-4f22-a02d-960d1594ff1c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:23.070468 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-df3ff298-b41d-4f22-a02d-960d1594ff1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6dca77e8b57479084b453d788f3d90b', 'request_id': 'req-df3ff298-b41d-4f22-a02d-960d1594ff1c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:23.077542 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-df3ff298-b41d-4f22-a02d-960d1594ff1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6dca77e8b57479084b453d788f3d90b', 'request_id': 'req-df3ff298-b41d-4f22-a02d-960d1594ff1c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:23.092312 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-df3ff298-b41d-4f22-a02d-960d1594ff1c admin admin] The token cache is being invalidate because project a6dca77e8b57479084b453d788f3d90b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:08:23.092619 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-df3ff298-b41d-4f22-a02d-960d1594ff1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:23.093328 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-df3ff298-b41d-4f22-a02d-960d1594ff1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:23.095072 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1442/2884] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:08:22 2022] DELETE /identity/v3/projects/a6dca77e8b57479084b453d788f3d90b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:08:23.101504 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b0fe6-662e-4e99-b36b-f262d1e4ee2d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:23.102074 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b0fe6-662e-4e99-b36b-f262d1e4ee2d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:23.102560 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b0fe6-662e-4e99-b36b-f262d1e4ee2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:23.125206 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-bc9b0fe6-662e-4e99-b36b-f262d1e4ee2d None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:08:23.126477 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bc9b0fe6-662e-4e99-b36b-f262d1e4ee2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:23.192761 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1443/2885] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:08:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 20:08:23.201866 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670f590-f9f1-4689-98d9-192f16c8e4db None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:23.202158 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670f590-f9f1-4689-98d9-192f16c8e4db None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:23.202441 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670f590-f9f1-4689-98d9-192f16c8e4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:23.222459 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a670f590-f9f1-4689-98d9-192f16c8e4db None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:08:23.223717 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a670f590-f9f1-4689-98d9-192f16c8e4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:23.234109 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1443/2886] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:08:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:08:23.238725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b0fe6-662e-4e99-b36b-f262d1e4ee2d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:23.249889 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b0fe6-662e-4e99-b36b-f262d1e4ee2d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:23.254487 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f8637-99da-4d6b-ae63-fa63e380af59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:23.255221 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f8637-99da-4d6b-ae63-fa63e380af59 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:23.255536 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f8637-99da-4d6b-ae63-fa63e380af59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:23.255811 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f8637-99da-4d6b-ae63-fa63e380af59 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:23.256545 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58f8637-99da-4d6b-ae63-fa63e380af59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 20:08:23.256850 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58f8637-99da-4d6b-ae63-fa63e380af59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:23.258519 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58f8637-99da-4d6b-ae63-fa63e380af59 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:23.271195 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1444/2887] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 20:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 20:08:23.276067 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670f590-f9f1-4689-98d9-192f16c8e4db None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:23.285483 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670f590-f9f1-4689-98d9-192f16c8e4db None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:23.291638 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b71f99-a115-468c-8063-0ad30c9b9614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:23.292311 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b71f99-a115-468c-8063-0ad30c9b9614 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:23.292604 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b71f99-a115-468c-8063-0ad30c9b9614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:23.292889 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b71f99-a115-468c-8063-0ad30c9b9614 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:23.293592 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b71f99-a115-468c-8063-0ad30c9b9614 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:23.295527 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b71f99-a115-468c-8063-0ad30c9b9614 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:23.323896 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1444/2888] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:08:23 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:08:23.328549 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f8637-99da-4d6b-ae63-fa63e380af59 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:23.339740 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f8637-99da-4d6b-ae63-fa63e380af59 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:23.400773 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254fecb-e13a-4e9e-a005-3379e4b0976b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:23.401592 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254fecb-e13a-4e9e-a005-3379e4b0976b admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:23.401953 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254fecb-e13a-4e9e-a005-3379e4b0976b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:23.402229 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254fecb-e13a-4e9e-a005-3379e4b0976b admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:23.402943 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5254fecb-e13a-4e9e-a005-3379e4b0976b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:23.405026 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5254fecb-e13a-4e9e-a005-3379e4b0976b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:23.427153 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1445/2889] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:08:23 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:08:23.432143 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b71f99-a115-468c-8063-0ad30c9b9614 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:23.442345 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b71f99-a115-468c-8063-0ad30c9b9614 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:23.447062 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd125723-6ba5-4921-82a9-4e47da3eede6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:23.447746 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd125723-6ba5-4921-82a9-4e47da3eede6 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:23.448032 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd125723-6ba5-4921-82a9-4e47da3eede6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:23.448309 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd125723-6ba5-4921-82a9-4e47da3eede6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:23.449037 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd125723-6ba5-4921-82a9-4e47da3eede6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:23.450642 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd125723-6ba5-4921-82a9-4e47da3eede6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:23.460855 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1445/2890] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:08:23.465393 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254fecb-e13a-4e9e-a005-3379e4b0976b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:23.475538 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254fecb-e13a-4e9e-a005-3379e4b0976b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:23.481076 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f7d78e-59bc-4d7d-9e25-8566cfd5faa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:23.481747 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7d78e-59bc-4d7d-9e25-8566cfd5faa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:23.482040 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7d78e-59bc-4d7d-9e25-8566cfd5faa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:23.482326 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7d78e-59bc-4d7d-9e25-8566cfd5faa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:23.483063 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f7d78e-59bc-4d7d-9e25-8566cfd5faa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:23.484722 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f7d78e-59bc-4d7d-9e25-8566cfd5faa9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:23.500477 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1446/2891] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:08:23.505892 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd125723-6ba5-4921-82a9-4e47da3eede6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:23.517985 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd125723-6ba5-4921-82a9-4e47da3eede6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:23.523006 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce37ea0-88fb-47cb-ad80-4c98c897f9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:23.523983 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce37ea0-88fb-47cb-ad80-4c98c897f9a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:23.524469 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce37ea0-88fb-47cb-ad80-4c98c897f9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:23.524971 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce37ea0-88fb-47cb-ad80-4c98c897f9a3 admin admin] PATH_INFO: `/v3/projects/f7aeadb9407449d7914d655cc649e942/users/4c433609381f41e4bc12c7d3d13b8d94/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:23.527683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:08:23.544020 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce37ea0-88fb-47cb-ad80-4c98c897f9a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7aeadb9407449d7914d655cc649e942, user_id=4c433609381f41e4bc12c7d3d13b8d94, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:23.546723 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce37ea0-88fb-47cb-ad80-4c98c897f9a3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:23.562286 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1446/2892] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:08:23 2022] PUT /identity/v3/projects/f7aeadb9407449d7914d655cc649e942/users/4c433609381f41e4bc12c7d3d13b8d94/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:08:23.572797 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cefa69-d220-4c74-9c68-3d9c0fb37cdd None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:23.573096 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cefa69-d220-4c74-9c68-3d9c0fb37cdd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:23.573379 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cefa69-d220-4c74-9c68-3d9c0fb37cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:23.582234 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:08:23.604329 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e2cefa69-d220-4c74-9c68-3d9c0fb37cdd None None] MFA Rules not processed for user `4c433609381f41e4bc12c7d3d13b8d94`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:08:23.605564 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e2cefa69-d220-4c74-9c68-3d9c0fb37cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:23.671647 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1447/2893] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:08:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:08:23.676331 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce37ea0-88fb-47cb-ad80-4c98c897f9a3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:23.740555 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce37ea0-88fb-47cb-ad80-4c98c897f9a3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:23.747695 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2dc5e-9a37-40a0-984e-728f1264013a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:23.748374 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2dc5e-9a37-40a0-984e-728f1264013a admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:23.748779 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2dc5e-9a37-40a0-984e-728f1264013a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:23.749072 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2dc5e-9a37-40a0-984e-728f1264013a admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:23.749908 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a2dc5e-9a37-40a0-984e-728f1264013a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:23.751848 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a2dc5e-9a37-40a0-984e-728f1264013a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:23.781213 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1447/2894] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:08:23 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:08:23.785555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cefa69-d220-4c74-9c68-3d9c0fb37cdd None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:23.796897 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cefa69-d220-4c74-9c68-3d9c0fb37cdd None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:23.856900 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8349308e-240f-4a7e-abda-1bba596e26fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:23.857493 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8349308e-240f-4a7e-abda-1bba596e26fd admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:23.857778 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8349308e-240f-4a7e-abda-1bba596e26fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:23.858054 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8349308e-240f-4a7e-abda-1bba596e26fd admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:23.858764 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8349308e-240f-4a7e-abda-1bba596e26fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:23.860752 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8349308e-240f-4a7e-abda-1bba596e26fd admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:23.885063 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1448/2895] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:08:23 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:08:23.889237 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2dc5e-9a37-40a0-984e-728f1264013a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:23.901473 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2dc5e-9a37-40a0-984e-728f1264013a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:23.907181 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081fd368-344e-47e4-8ade-ae147149e697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:23.907948 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fd368-344e-47e4-8ade-ae147149e697 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:23.908326 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fd368-344e-47e4-8ade-ae147149e697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:23.908688 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fd368-344e-47e4-8ade-ae147149e697 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:23.909481 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081fd368-344e-47e4-8ade-ae147149e697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:23.911259 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081fd368-344e-47e4-8ade-ae147149e697 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:23.925258 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1448/2896] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:08:23.929425 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8349308e-240f-4a7e-abda-1bba596e26fd admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:23.942206 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8349308e-240f-4a7e-abda-1bba596e26fd admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:23.948725 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e9b478-455f-4a0c-abca-419c3c02ad06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:23.949398 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e9b478-455f-4a0c-abca-419c3c02ad06 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:23.949685 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e9b478-455f-4a0c-abca-419c3c02ad06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:23.949994 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e9b478-455f-4a0c-abca-419c3c02ad06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:23.950732 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e9b478-455f-4a0c-abca-419c3c02ad06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:23.952455 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e9b478-455f-4a0c-abca-419c3c02ad06 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:23.964129 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1449/2897] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:08:23.968535 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081fd368-344e-47e4-8ade-ae147149e697 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:23.980517 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081fd368-344e-47e4-8ade-ae147149e697 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:23.985594 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56cecf0-b28a-4efc-a327-3d0fda8765d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:23.986496 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56cecf0-b28a-4efc-a327-3d0fda8765d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:23.986902 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56cecf0-b28a-4efc-a327-3d0fda8765d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:23.987099 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56cecf0-b28a-4efc-a327-3d0fda8765d2 admin admin] PATH_INFO: `/v3/projects/f9f0a664862a4e608c12194ea758a0f0/users/b01e258d403f4334a6483e52033c3ea5/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:24.005109 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56cecf0-b28a-4efc-a327-3d0fda8765d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9f0a664862a4e608c12194ea758a0f0, user_id=b01e258d403f4334a6483e52033c3ea5, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:24.007914 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56cecf0-b28a-4efc-a327-3d0fda8765d2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:24.021345 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1449/2898] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:08:23 2022] PUT /identity/v3/projects/f9f0a664862a4e608c12194ea758a0f0/users/b01e258d403f4334a6483e52033c3ea5/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:08:24.032222 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c3dcc-2363-445b-acb6-09d297439b9f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:24.032517 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c3dcc-2363-445b-acb6-09d297439b9f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:24.032803 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c3dcc-2363-445b-acb6-09d297439b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:24.062608 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-722c3dcc-2363-445b-acb6-09d297439b9f None None] MFA Rules not processed for user `b01e258d403f4334a6483e52033c3ea5`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:08:24.063858 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-722c3dcc-2363-445b-acb6-09d297439b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:24.134257 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1450/2899] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:08:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:08:24.138772 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56cecf0-b28a-4efc-a327-3d0fda8765d2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:24.207281 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56cecf0-b28a-4efc-a327-3d0fda8765d2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:24.212222 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd060db-a767-4547-abbc-3332b6309dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:24.212897 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd060db-a767-4547-abbc-3332b6309dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:24.213184 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd060db-a767-4547-abbc-3332b6309dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:24.213458 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd060db-a767-4547-abbc-3332b6309dc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:24.214155 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd060db-a767-4547-abbc-3332b6309dc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:24.216107 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd060db-a767-4547-abbc-3332b6309dc0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:24.245421 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1450/2900] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:08:24 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:08:24.249499 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722c3dcc-2363-445b-acb6-09d297439b9f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:24.255082 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:08:24.262498 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722c3dcc-2363-445b-acb6-09d297439b9f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:24.325505 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c585b5-7727-43c2-b02a-64e32276a05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:24.326391 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c585b5-7727-43c2-b02a-64e32276a05e admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:24.326766 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c585b5-7727-43c2-b02a-64e32276a05e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:24.327148 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c585b5-7727-43c2-b02a-64e32276a05e admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:24.327979 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c585b5-7727-43c2-b02a-64e32276a05e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:24.330447 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c585b5-7727-43c2-b02a-64e32276a05e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:24.355872 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1451/2901] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:08:24 2022] POST /identity/v3/users => generated 298 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:08:24.360030 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd060db-a767-4547-abbc-3332b6309dc0 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:24.371467 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd060db-a767-4547-abbc-3332b6309dc0 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:24.376313 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ebd09-8628-43c8-9d61-d69692810f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:24.376982 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ebd09-8628-43c8-9d61-d69692810f0d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:24.377266 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ebd09-8628-43c8-9d61-d69692810f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:24.377668 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ebd09-8628-43c8-9d61-d69692810f0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:24.378899 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ebd09-8628-43c8-9d61-d69692810f0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:24.382086 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ebd09-8628-43c8-9d61-d69692810f0d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:24.395841 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1451/2902] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:08:24.400526 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c585b5-7727-43c2-b02a-64e32276a05e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:24.410854 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c585b5-7727-43c2-b02a-64e32276a05e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:24.416235 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e75561-d7bd-4341-8a65-5f4fafed83db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:24.417158 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e75561-d7bd-4341-8a65-5f4fafed83db admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:24.417437 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e75561-d7bd-4341-8a65-5f4fafed83db admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:24.417718 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e75561-d7bd-4341-8a65-5f4fafed83db admin admin] PATH_INFO: `/v3/projects/66aefbd1a8244317a78aec099ce655a7/users/e4e1052349ce4bfd9fe50f1056c71cec/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:24.435177 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e75561-d7bd-4341-8a65-5f4fafed83db admin admin] RBAC: Authorizing `identity:create_grant(project_id=66aefbd1a8244317a78aec099ce655a7, user_id=e4e1052349ce4bfd9fe50f1056c71cec, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:24.437889 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e75561-d7bd-4341-8a65-5f4fafed83db admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:24.452732 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1452/2903] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:08:24 2022] PUT /identity/v3/projects/66aefbd1a8244317a78aec099ce655a7/users/e4e1052349ce4bfd9fe50f1056c71cec/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:08:24.457354 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ebd09-8628-43c8-9d61-d69692810f0d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:24.469126 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ebd09-8628-43c8-9d61-d69692810f0d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:24.472924 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:08:24.531681 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec2e91-aca3-4330-9353-f6e31fcd3069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:24.532539 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec2e91-aca3-4330-9353-f6e31fcd3069 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:24.532857 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec2e91-aca3-4330-9353-f6e31fcd3069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:24.533138 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec2e91-aca3-4330-9353-f6e31fcd3069 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:24.533901 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec2e91-aca3-4330-9353-f6e31fcd3069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:24.535546 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec2e91-aca3-4330-9353-f6e31fcd3069 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:24.548334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1452/2904] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:08:24.552803 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e75561-d7bd-4341-8a65-5f4fafed83db admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:24.563970 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e75561-d7bd-4341-8a65-5f4fafed83db admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:24.568874 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5b2f9-c1ae-483f-8553-ce7fd1ec3236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:24.569788 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5b2f9-c1ae-483f-8553-ce7fd1ec3236 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:24.570069 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5b2f9-c1ae-483f-8553-ce7fd1ec3236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:24.570334 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5b2f9-c1ae-483f-8553-ce7fd1ec3236 admin admin] PATH_INFO: `/v3/projects/66aefbd1a8244317a78aec099ce655a7/users/e4e1052349ce4bfd9fe50f1056c71cec/roles/ed7fec18968241efa76a0ee769d96da9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:24.572518 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5b2f9-c1ae-483f-8553-ce7fd1ec3236 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66aefbd1a8244317a78aec099ce655a7, user_id=e4e1052349ce4bfd9fe50f1056c71cec, role_id=ed7fec18968241efa76a0ee769d96da9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:24.575134 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5b2f9-c1ae-483f-8553-ce7fd1ec3236 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:24.587604 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1453/2905] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:08:24 2022] PUT /identity/v3/projects/66aefbd1a8244317a78aec099ce655a7/users/e4e1052349ce4bfd9fe50f1056c71cec/roles/ed7fec18968241efa76a0ee769d96da9 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:08:24.599061 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b9e37-d839-4a73-aaa5-871417d55417 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:24.599356 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b9e37-d839-4a73-aaa5-871417d55417 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:24.599670 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b9e37-d839-4a73-aaa5-871417d55417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:24.629361 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c77b9e37-d839-4a73-aaa5-871417d55417 None None] MFA Rules not processed for user `e4e1052349ce4bfd9fe50f1056c71cec`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:08:24.630584 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c77b9e37-d839-4a73-aaa5-871417d55417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:24.703826 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1453/2906] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:08:24 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 20:08:24.716104 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5b2f9-c1ae-483f-8553-ce7fd1ec3236 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:24.717841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-08f5b2f9-c1ae-483f-8553-ce7fd1ec3236 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:24.794237 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5b2f9-c1ae-483f-8553-ce7fd1ec3236 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:24.797994 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d5cdd7-e7d3-4b3c-b867-d86715f2b53d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:24.798647 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5cdd7-e7d3-4b3c-b867-d86715f2b53d service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:24.798930 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5cdd7-e7d3-4b3c-b867-d86715f2b53d service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:24.799215 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5cdd7-e7d3-4b3c-b867-d86715f2b53d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:24.860411 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d5cdd7-e7d3-4b3c-b867-d86715f2b53d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:24.861872 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d5cdd7-e7d3-4b3c-b867-d86715f2b53d service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:24.865519 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:08:24.874042 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1454/2907] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:08:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:08:26.421975 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b9e37-d839-4a73-aaa5-871417d55417 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:26.423402 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c77b9e37-d839-4a73-aaa5-871417d55417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:26.435846 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b9e37-d839-4a73-aaa5-871417d55417 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:26.439492 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd1e28-1436-4646-b57d-0f41f1f6814b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:26.440177 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd1e28-1436-4646-b57d-0f41f1f6814b service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:26.440497 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd1e28-1436-4646-b57d-0f41f1f6814b service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:26.440776 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd1e28-1436-4646-b57d-0f41f1f6814b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:26.507281 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfd1e28-1436-4646-b57d-0f41f1f6814b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:26.509821 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfd1e28-1436-4646-b57d-0f41f1f6814b service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:26.520160 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1454/2908] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:08:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:08:27.144523 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d5cdd7-e7d3-4b3c-b867-d86715f2b53d service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:27.161700 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d5cdd7-e7d3-4b3c-b867-d86715f2b53d service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:27.169053 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c2385b-7d7e-44f8-b201-6771193b042e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:27.170247 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2385b-7d7e-44f8-b201-6771193b042e service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:27.171611 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2385b-7d7e-44f8-b201-6771193b042e service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:27.171611 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2385b-7d7e-44f8-b201-6771193b042e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:27.183938 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c2385b-7d7e-44f8-b201-6771193b042e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:27.185415 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c2385b-7d7e-44f8-b201-6771193b042e service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:27.197246 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1455/2909] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:08:27 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:08:31.130602 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd1e28-1436-4646-b57d-0f41f1f6814b service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:31.161567 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:08:31.200949 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd1e28-1436-4646-b57d-0f41f1f6814b service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:31.206341 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660da10-a066-4c19-bb32-bd0d6221fc2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:31.207219 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660da10-a066-4c19-bb32-bd0d6221fc2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:31.207501 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660da10-a066-4c19-bb32-bd0d6221fc2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:31.207810 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660da10-a066-4c19-bb32-bd0d6221fc2f admin admin] PATH_INFO: `/v3/users/4c433609381f41e4bc12c7d3d13b8d94` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:31.209371 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c660da10-a066-4c19-bb32-bd0d6221fc2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c433609381f41e4bc12c7d3d13b8d94)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:31.211383 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c660da10-a066-4c19-bb32-bd0d6221fc2f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:31.262104 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c660da10-a066-4c19-bb32-bd0d6221fc2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c433609381f41e4bc12c7d3d13b8d94'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:31.269105 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c660da10-a066-4c19-bb32-bd0d6221fc2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c433609381f41e4bc12c7d3d13b8d94'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:31.277133 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c660da10-a066-4c19-bb32-bd0d6221fc2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c433609381f41e4bc12c7d3d13b8d94'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:31.277763 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c660da10-a066-4c19-bb32-bd0d6221fc2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c433609381f41e4bc12c7d3d13b8d94'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:31.278378 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c660da10-a066-4c19-bb32-bd0d6221fc2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c433609381f41e4bc12c7d3d13b8d94'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:31.296140 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1455/2910] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:08:31 2022] DELETE /identity/v3/users/4c433609381f41e4bc12c7d3d13b8d94 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:08:31.300762 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230fe2f-82c4-4bc2-9309-610167f1d56c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:31.301244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230fe2f-82c4-4bc2-9309-610167f1d56c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:31.301441 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230fe2f-82c4-4bc2-9309-610167f1d56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:31.330677 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2230fe2f-82c4-4bc2-9309-610167f1d56c None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:08:31.332069 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2230fe2f-82c4-4bc2-9309-610167f1d56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:31.392024 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1456/2911] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:08:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:08:31.402478 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660da10-a066-4c19-bb32-bd0d6221fc2f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:31.403920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c660da10-a066-4c19-bb32-bd0d6221fc2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:31.463077 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660da10-a066-4c19-bb32-bd0d6221fc2f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:31.466678 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0353c5e-4530-4f1b-88f3-46ef3501458a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:31.467329 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0353c5e-4530-4f1b-88f3-46ef3501458a service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:31.467644 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0353c5e-4530-4f1b-88f3-46ef3501458a service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:31.467924 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0353c5e-4530-4f1b-88f3-46ef3501458a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:31.477476 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0353c5e-4530-4f1b-88f3-46ef3501458a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:31.478915 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0353c5e-4530-4f1b-88f3-46ef3501458a service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:31.488652 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1456/2912] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:08:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:08:31.766590 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230fe2f-82c4-4bc2-9309-610167f1d56c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:31.768105 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2230fe2f-82c4-4bc2-9309-610167f1d56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:31.782457 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230fe2f-82c4-4bc2-9309-610167f1d56c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:31.789216 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab0c23-ff6b-4eac-ba35-335c3a6c05f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:31.790322 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab0c23-ff6b-4eac-ba35-335c3a6c05f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:31.790776 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab0c23-ff6b-4eac-ba35-335c3a6c05f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:31.791164 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab0c23-ff6b-4eac-ba35-335c3a6c05f3 admin admin] PATH_INFO: `/v3/projects/f7aeadb9407449d7914d655cc649e942` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:31.792822 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ab0c23-ff6b-4eac-ba35-335c3a6c05f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7aeadb9407449d7914d655cc649e942)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:31.795615 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ab0c23-ff6b-4eac-ba35-335c3a6c05f3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:31.850065 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-13ab0c23-ff6b-4eac-ba35-335c3a6c05f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7aeadb9407449d7914d655cc649e942', 'request_id': 'req-13ab0c23-ff6b-4eac-ba35-335c3a6c05f3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:31.858213 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-13ab0c23-ff6b-4eac-ba35-335c3a6c05f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7aeadb9407449d7914d655cc649e942', 'request_id': 'req-13ab0c23-ff6b-4eac-ba35-335c3a6c05f3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:31.864142 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-13ab0c23-ff6b-4eac-ba35-335c3a6c05f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7aeadb9407449d7914d655cc649e942', 'request_id': 'req-13ab0c23-ff6b-4eac-ba35-335c3a6c05f3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:31.876114 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-13ab0c23-ff6b-4eac-ba35-335c3a6c05f3 admin admin] The token cache is being invalidate because project f7aeadb9407449d7914d655cc649e942 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:08:31.876399 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-13ab0c23-ff6b-4eac-ba35-335c3a6c05f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:31.877039 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-13ab0c23-ff6b-4eac-ba35-335c3a6c05f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:31.878580 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1457/2913] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:08:31 2022] DELETE /identity/v3/projects/f7aeadb9407449d7914d655cc649e942 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:08:31.882562 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0353c5e-4530-4f1b-88f3-46ef3501458a service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:31.892779 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0353c5e-4530-4f1b-88f3-46ef3501458a service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:31.937808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:08:31.952186 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdac19d-e61f-4ad5-97b5-6fe5c3ec4813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:31.953064 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdac19d-e61f-4ad5-97b5-6fe5c3ec4813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:31.953343 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdac19d-e61f-4ad5-97b5-6fe5c3ec4813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:31.953617 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdac19d-e61f-4ad5-97b5-6fe5c3ec4813 admin admin] PATH_INFO: `/v3/users/b01e258d403f4334a6483e52033c3ea5` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:31.954808 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdac19d-e61f-4ad5-97b5-6fe5c3ec4813 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b01e258d403f4334a6483e52033c3ea5)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:31.956780 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdac19d-e61f-4ad5-97b5-6fe5c3ec4813 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:32.002474 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-abdac19d-e61f-4ad5-97b5-6fe5c3ec4813 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b01e258d403f4334a6483e52033c3ea5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:32.011255 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-abdac19d-e61f-4ad5-97b5-6fe5c3ec4813 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b01e258d403f4334a6483e52033c3ea5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:32.018412 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-abdac19d-e61f-4ad5-97b5-6fe5c3ec4813 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b01e258d403f4334a6483e52033c3ea5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:32.019069 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-abdac19d-e61f-4ad5-97b5-6fe5c3ec4813 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b01e258d403f4334a6483e52033c3ea5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:32.019737 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-abdac19d-e61f-4ad5-97b5-6fe5c3ec4813 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b01e258d403f4334a6483e52033c3ea5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:32.038130 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1457/2914] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:08:31 2022] DELETE /identity/v3/users/b01e258d403f4334a6483e52033c3ea5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 20:08:32.340840 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab0c23-ff6b-4eac-ba35-335c3a6c05f3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:32.342412 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-13ab0c23-ff6b-4eac-ba35-335c3a6c05f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:32.361360 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab0c23-ff6b-4eac-ba35-335c3a6c05f3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:32.423906 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd1307-d4a1-4727-ac31-9be471fcb82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:32.424775 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd1307-d4a1-4727-ac31-9be471fcb82c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:32.425059 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd1307-d4a1-4727-ac31-9be471fcb82c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:32.425336 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd1307-d4a1-4727-ac31-9be471fcb82c admin admin] PATH_INFO: `/v3/projects/f9f0a664862a4e608c12194ea758a0f0` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:32.426540 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbd1307-d4a1-4727-ac31-9be471fcb82c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9f0a664862a4e608c12194ea758a0f0)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:32.428503 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbd1307-d4a1-4727-ac31-9be471fcb82c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:32.430903 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:08:32.478527 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ddbd1307-d4a1-4727-ac31-9be471fcb82c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9f0a664862a4e608c12194ea758a0f0', 'request_id': 'req-ddbd1307-d4a1-4727-ac31-9be471fcb82c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:32.486145 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ddbd1307-d4a1-4727-ac31-9be471fcb82c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9f0a664862a4e608c12194ea758a0f0', 'request_id': 'req-ddbd1307-d4a1-4727-ac31-9be471fcb82c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:32.492188 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ddbd1307-d4a1-4727-ac31-9be471fcb82c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9f0a664862a4e608c12194ea758a0f0', 'request_id': 'req-ddbd1307-d4a1-4727-ac31-9be471fcb82c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:32.504294 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ddbd1307-d4a1-4727-ac31-9be471fcb82c admin admin] The token cache is being invalidate because project f9f0a664862a4e608c12194ea758a0f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:08:32.504588 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ddbd1307-d4a1-4727-ac31-9be471fcb82c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:32.505244 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ddbd1307-d4a1-4727-ac31-9be471fcb82c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:32.506785 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1458/2915] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:08:32 2022] DELETE /identity/v3/projects/f9f0a664862a4e608c12194ea758a0f0 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:08:32.511874 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdac19d-e61f-4ad5-97b5-6fe5c3ec4813 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:32.513309 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-abdac19d-e61f-4ad5-97b5-6fe5c3ec4813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:32.527260 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdac19d-e61f-4ad5-97b5-6fe5c3ec4813 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:32.586050 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7fa77-7dff-4d99-9a81-fa7c5ad4a011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:32.587015 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7fa77-7dff-4d99-9a81-fa7c5ad4a011 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:32.587294 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7fa77-7dff-4d99-9a81-fa7c5ad4a011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:32.587596 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7fa77-7dff-4d99-9a81-fa7c5ad4a011 admin admin] PATH_INFO: `/v3/users/e4e1052349ce4bfd9fe50f1056c71cec` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:32.588787 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e7fa77-7dff-4d99-9a81-fa7c5ad4a011 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4e1052349ce4bfd9fe50f1056c71cec)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:32.590707 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e7fa77-7dff-4d99-9a81-fa7c5ad4a011 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:32.639457 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-53e7fa77-7dff-4d99-9a81-fa7c5ad4a011 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4e1052349ce4bfd9fe50f1056c71cec'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:32.646320 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-53e7fa77-7dff-4d99-9a81-fa7c5ad4a011 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4e1052349ce4bfd9fe50f1056c71cec'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:32.653947 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-53e7fa77-7dff-4d99-9a81-fa7c5ad4a011 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4e1052349ce4bfd9fe50f1056c71cec'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:32.654855 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-53e7fa77-7dff-4d99-9a81-fa7c5ad4a011 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4e1052349ce4bfd9fe50f1056c71cec'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:32.655463 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-53e7fa77-7dff-4d99-9a81-fa7c5ad4a011 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4e1052349ce4bfd9fe50f1056c71cec'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:32.660875 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:08:32.674468 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1458/2916] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:08:32 2022] DELETE /identity/v3/users/e4e1052349ce4bfd9fe50f1056c71cec => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:08:32.959412 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd1307-d4a1-4727-ac31-9be471fcb82c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:32.960992 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ddbd1307-d4a1-4727-ac31-9be471fcb82c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:32.981078 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd1307-d4a1-4727-ac31-9be471fcb82c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:33.043446 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba52d68-83ea-4d0d-a044-609e87253771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:33.044350 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba52d68-83ea-4d0d-a044-609e87253771 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:33.044629 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba52d68-83ea-4d0d-a044-609e87253771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:33.044907 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba52d68-83ea-4d0d-a044-609e87253771 admin admin] PATH_INFO: `/v3/projects/66aefbd1a8244317a78aec099ce655a7` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:33.046332 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba52d68-83ea-4d0d-a044-609e87253771 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66aefbd1a8244317a78aec099ce655a7)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:33.048869 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba52d68-83ea-4d0d-a044-609e87253771 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:33.101335 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4ba52d68-83ea-4d0d-a044-609e87253771 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66aefbd1a8244317a78aec099ce655a7', 'request_id': 'req-4ba52d68-83ea-4d0d-a044-609e87253771'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:33.112328 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4ba52d68-83ea-4d0d-a044-609e87253771 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66aefbd1a8244317a78aec099ce655a7', 'request_id': 'req-4ba52d68-83ea-4d0d-a044-609e87253771'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:33.123005 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4ba52d68-83ea-4d0d-a044-609e87253771 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66aefbd1a8244317a78aec099ce655a7', 'request_id': 'req-4ba52d68-83ea-4d0d-a044-609e87253771'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:33.135067 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4ba52d68-83ea-4d0d-a044-609e87253771 admin admin] The token cache is being invalidate because project 66aefbd1a8244317a78aec099ce655a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:08:33.135318 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4ba52d68-83ea-4d0d-a044-609e87253771 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:33.136324 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4ba52d68-83ea-4d0d-a044-609e87253771 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:08:33.138118 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1459/2917] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:08:32 2022] DELETE /identity/v3/projects/66aefbd1a8244317a78aec099ce655a7 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:08:33.143157 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6c225-fe20-4a88-84cf-5744b0aa59d6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:33.143451 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6c225-fe20-4a88-84cf-5744b0aa59d6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:33.143759 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6c225-fe20-4a88-84cf-5744b0aa59d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:33.200440 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-33b6c225-fe20-4a88-84cf-5744b0aa59d6 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:08:33.201632 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-33b6c225-fe20-4a88-84cf-5744b0aa59d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:33.269995 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1459/2918] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:08:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:08:33.278190 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e88ec0-f336-44c7-ac42-a533c1e54502 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:33.278482 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e88ec0-f336-44c7-ac42-a533c1e54502 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:33.278763 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e88ec0-f336-44c7-ac42-a533c1e54502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:33.301526 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-79e88ec0-f336-44c7-ac42-a533c1e54502 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:08:33.302768 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-79e88ec0-f336-44c7-ac42-a533c1e54502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:33.313517 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1460/2919] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:08:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 20:08:33.317868 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b6c225-fe20-4a88-84cf-5744b0aa59d6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:33.329062 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b6c225-fe20-4a88-84cf-5744b0aa59d6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:33.333750 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf201811-4db0-48dd-8fe7-663eee302530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:33.334416 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf201811-4db0-48dd-8fe7-663eee302530 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:33.334718 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf201811-4db0-48dd-8fe7-663eee302530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:33.334996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf201811-4db0-48dd-8fe7-663eee302530 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:33.335750 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf201811-4db0-48dd-8fe7-663eee302530 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 20:08:33.336065 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf201811-4db0-48dd-8fe7-663eee302530 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:33.339153 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf201811-4db0-48dd-8fe7-663eee302530 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:33.351590 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1460/2920] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 20:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 20:08:33.357619 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e88ec0-f336-44c7-ac42-a533c1e54502 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:33.366235 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e88ec0-f336-44c7-ac42-a533c1e54502 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:33.372224 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e2956-d974-425e-aee6-c5c6dd008f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:33.372893 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e2956-d974-425e-aee6-c5c6dd008f77 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:33.373172 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e2956-d974-425e-aee6-c5c6dd008f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:33.373449 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e2956-d974-425e-aee6-c5c6dd008f77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:33.374148 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61e2956-d974-425e-aee6-c5c6dd008f77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:33.376111 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61e2956-d974-425e-aee6-c5c6dd008f77 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:33.392912 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:08:33.423923 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1461/2921] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:08:33 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 20:08:33.428175 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf201811-4db0-48dd-8fe7-663eee302530 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:33.439439 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf201811-4db0-48dd-8fe7-663eee302530 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:33.505947 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247de48-8edd-4349-a132-66a181e62878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:33.506911 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247de48-8edd-4349-a132-66a181e62878 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:33.507271 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247de48-8edd-4349-a132-66a181e62878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:33.507575 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247de48-8edd-4349-a132-66a181e62878 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:33.508400 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c247de48-8edd-4349-a132-66a181e62878 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:33.510364 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c247de48-8edd-4349-a132-66a181e62878 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:33.533750 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1461/2922] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:08:33 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:08:33.538108 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e2956-d974-425e-aee6-c5c6dd008f77 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:33.550288 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e2956-d974-425e-aee6-c5c6dd008f77 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:33.555447 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07fc97b-19af-4bae-84a3-3573101464e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:33.556191 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07fc97b-19af-4bae-84a3-3573101464e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:33.556428 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07fc97b-19af-4bae-84a3-3573101464e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:33.556705 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07fc97b-19af-4bae-84a3-3573101464e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:33.557438 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07fc97b-19af-4bae-84a3-3573101464e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:33.559347 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07fc97b-19af-4bae-84a3-3573101464e4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:33.570224 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1462/2923] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:08:33.574823 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247de48-8edd-4349-a132-66a181e62878 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:33.585388 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247de48-8edd-4349-a132-66a181e62878 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:33.590725 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a0640d-bf14-43fd-8b98-e88ca6c2e232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:33.591398 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0640d-bf14-43fd-8b98-e88ca6c2e232 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:33.591715 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0640d-bf14-43fd-8b98-e88ca6c2e232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:33.591995 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0640d-bf14-43fd-8b98-e88ca6c2e232 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:33.592725 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a0640d-bf14-43fd-8b98-e88ca6c2e232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:33.594392 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a0640d-bf14-43fd-8b98-e88ca6c2e232 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:33.606707 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1462/2924] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:08:33.610833 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07fc97b-19af-4bae-84a3-3573101464e4 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:33.624108 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07fc97b-19af-4bae-84a3-3573101464e4 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:33.628865 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e10c5c-06a3-4d32-8122-e92b71e6c685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:33.629755 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e10c5c-06a3-4d32-8122-e92b71e6c685 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:33.630099 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e10c5c-06a3-4d32-8122-e92b71e6c685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:33.630389 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e10c5c-06a3-4d32-8122-e92b71e6c685 admin admin] PATH_INFO: `/v3/projects/6ca50c708922486a910456429af51e2e/users/64c8111203d34490a0db95a739f73e0b/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:33.646016 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e10c5c-06a3-4d32-8122-e92b71e6c685 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ca50c708922486a910456429af51e2e, user_id=64c8111203d34490a0db95a739f73e0b, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:33.649467 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e10c5c-06a3-4d32-8122-e92b71e6c685 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:33.663208 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1463/2925] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:08:33 2022] PUT /identity/v3/projects/6ca50c708922486a910456429af51e2e/users/64c8111203d34490a0db95a739f73e0b/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:08:33.669526 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeba6f3-15dd-42c0-b017-0775a1c2d22f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:33.669816 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeba6f3-15dd-42c0-b017-0775a1c2d22f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:33.670096 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeba6f3-15dd-42c0-b017-0775a1c2d22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:33.689765 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-cfeba6f3-15dd-42c0-b017-0775a1c2d22f None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:08:33.690989 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cfeba6f3-15dd-42c0-b017-0775a1c2d22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:33.728026 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:08:33.768119 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1463/2926] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:08:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:08:33.778835 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e10c5c-06a3-4d32-8122-e92b71e6c685 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:33.780632 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a6e10c5c-06a3-4d32-8122-e92b71e6c685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:33.847831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e10c5c-06a3-4d32-8122-e92b71e6c685 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:33.851520 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28323702-a771-4cf4-a95c-37931c3ddf3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:33.852330 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28323702-a771-4cf4-a95c-37931c3ddf3c service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:33.852613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28323702-a771-4cf4-a95c-37931c3ddf3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:33.852894 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28323702-a771-4cf4-a95c-37931c3ddf3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:33.862470 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28323702-a771-4cf4-a95c-37931c3ddf3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:33.864037 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28323702-a771-4cf4-a95c-37931c3ddf3c service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:33.874362 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1464/2927] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:08:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:08:34.775277 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d28b3-7b74-4bb8-b2f4-c961e8df3365 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:34.775866 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d28b3-7b74-4bb8-b2f4-c961e8df3365 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:34.776082 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d28b3-7b74-4bb8-b2f4-c961e8df3365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:34.805770 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c71d28b3-7b74-4bb8-b2f4-c961e8df3365 None None] MFA Rules not processed for user `64c8111203d34490a0db95a739f73e0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:08:34.807126 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c71d28b3-7b74-4bb8-b2f4-c961e8df3365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:34.879860 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1464/2928] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:08:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:08:34.890851 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28323702-a771-4cf4-a95c-37931c3ddf3c service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:34.892276 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-28323702-a771-4cf4-a95c-37931c3ddf3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:34.955566 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28323702-a771-4cf4-a95c-37931c3ddf3c service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:34.959558 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403ff92-8b6d-4aae-bfe2-54132438716a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:34.960393 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403ff92-8b6d-4aae-bfe2-54132438716a service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:34.960845 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403ff92-8b6d-4aae-bfe2-54132438716a service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:34.961651 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403ff92-8b6d-4aae-bfe2-54132438716a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:34.971096 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4403ff92-8b6d-4aae-bfe2-54132438716a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:34.972601 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4403ff92-8b6d-4aae-bfe2-54132438716a service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:34.982673 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1465/2929] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:08:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:08:36.373946 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fadc9-8fcc-486a-be80-bccbc85bafa3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:36.374449 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fadc9-8fcc-486a-be80-bccbc85bafa3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:36.374530 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fadc9-8fcc-486a-be80-bccbc85bafa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:36.394787 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-809fadc9-8fcc-486a-be80-bccbc85bafa3 None None] MFA Rules not processed for user `64c8111203d34490a0db95a739f73e0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:08:36.396062 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-809fadc9-8fcc-486a-be80-bccbc85bafa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:36.404703 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1465/2930] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:08:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:08:36.661021 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403ff92-8b6d-4aae-bfe2-54132438716a service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:36.662508 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4403ff92-8b6d-4aae-bfe2-54132438716a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:36.675146 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403ff92-8b6d-4aae-bfe2-54132438716a service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:36.678621 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f45f4e-f8c4-4f5a-977f-046ee098f9e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:36.679257 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f45f4e-f8c4-4f5a-977f-046ee098f9e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:36.679569 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f45f4e-f8c4-4f5a-977f-046ee098f9e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:36.679856 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f45f4e-f8c4-4f5a-977f-046ee098f9e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:36.681657 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-18f45f4e-f8c4-4f5a-977f-046ee098f9e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:36.691064 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f45f4e-f8c4-4f5a-977f-046ee098f9e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:36.692590 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f45f4e-f8c4-4f5a-977f-046ee098f9e3 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:36.701430 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1466/2931] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:08:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:08:46.389381 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=49 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:08:46.697674 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=79,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:08:52.002732 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809fadc9-8fcc-486a-be80-bccbc85bafa3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:08:52.004467 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-809fadc9-8fcc-486a-be80-bccbc85bafa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:52.100410 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809fadc9-8fcc-486a-be80-bccbc85bafa3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:08:52.117936 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8511692-578d-4c01-84a5-013c3dd51a2d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0422d6695bf14925bf0f993cb16e64ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:08:52.119080 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8511692-578d-4c01-84a5-013c3dd51a2d service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:52.119608 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8511692-578d-4c01-84a5-013c3dd51a2d service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:52.120115 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8511692-578d-4c01-84a5-013c3dd51a2d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:52.123277 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e8511692-578d-4c01-84a5-013c3dd51a2d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:52.138861 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8511692-578d-4c01-84a5-013c3dd51a2d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:08:52.140606 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8511692-578d-4c01-84a5-013c3dd51a2d service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:08:52.151761 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1466/2932] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:08:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:08:54.374805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b11b29-9490-4940-a58d-a198aa0436c8 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:08:54.375327 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b11b29-9490-4940-a58d-a198aa0436c8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:08:54.375421 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b11b29-9490-4940-a58d-a198aa0436c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:08:54.394523 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-45b11b29-9490-4940-a58d-a198aa0436c8 None None] MFA Rules not processed for user `64c8111203d34490a0db95a739f73e0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:08:54.395933 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-45b11b29-9490-4940-a58d-a198aa0436c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:08:54.403344 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1467/2933] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:08:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:09:02.146668 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=37 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:09:04.389289 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:09:16.932030 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9ece5-9705-4694-a852-dd26878467d9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:09:16.932568 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9ece5-9705-4694-a852-dd26878467d9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:09:16.932568 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9ece5-9705-4694-a852-dd26878467d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:09:16.953603 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-12a9ece5-9705-4694-a852-dd26878467d9 None None] MFA Rules not processed for user `1c0772205ae24713b8901666a87018b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:09:16.954827 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-12a9ece5-9705-4694-a852-dd26878467d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:09:17.012007 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1467/2934] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Jul 25 20:09:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:09:17.061613 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b11b29-9490-4940-a58d-a198aa0436c8 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:09:17.063229 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-45b11b29-9490-4940-a58d-a198aa0436c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:09:17.136449 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b11b29-9490-4940-a58d-a198aa0436c8 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:09:17.142451 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc1613-125d-4a96-a695-cbdea1ca2174 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feafcf1b5a9844fe9738823fddd07e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:09:17.143138 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc1613-125d-4a96-a695-cbdea1ca2174 service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:09:17.143429 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc1613-125d-4a96-a695-cbdea1ca2174 service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:09:17.143753 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc1613-125d-4a96-a695-cbdea1ca2174 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:09:17.152445 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc1613-125d-4a96-a695-cbdea1ca2174 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:09:17.154872 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc1613-125d-4a96-a695-cbdea1ca2174 service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:09:17.163094 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1468/2935] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Jul 25 20:09:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 20:09:27.005131 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=25 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:09:27.159625 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=32 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:09:32.629141 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdcf46-cd06-4fd9-a7c9-9bc0180a3fe9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:09:32.629652 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdcf46-cd06-4fd9-a7c9-9bc0180a3fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:09:32.629756 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdcf46-cd06-4fd9-a7c9-9bc0180a3fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:09:32.649321 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-97fdcf46-cd06-4fd9-a7c9-9bc0180a3fe9 None None] MFA Rules not processed for user `1c0772205ae24713b8901666a87018b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:09:32.650575 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-97fdcf46-cd06-4fd9-a7c9-9bc0180a3fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:09:32.658002 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1468/2936] 10.222.0.18 () {58 vars in 967 bytes} [Mon Jul 25 20:09:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 20:09:32.670294 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc1613-125d-4a96-a695-cbdea1ca2174 service swift] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:09:32.681154 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc1613-125d-4a96-a695-cbdea1ca2174 service swift] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:09:32.686875 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102a1f2-9cb9-43bb-92b6-12ca638706ab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'feafcf1b5a9844fe9738823fddd07e0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:09:32.687528 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102a1f2-9cb9-43bb-92b6-12ca638706ab service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:09:32.687817 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102a1f2-9cb9-43bb-92b6-12ca638706ab service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:09:32.688096 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102a1f2-9cb9-43bb-92b6-12ca638706ab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:09:32.696359 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5102a1f2-9cb9-43bb-92b6-12ca638706ab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:09:32.697938 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5102a1f2-9cb9-43bb-92b6-12ca638706ab service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:09:32.706906 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1469/2937] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Jul 25 20:09:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 20:09:34.695070 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fdcf46-cd06-4fd9-a7c9-9bc0180a3fe9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:09:34.707995 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fdcf46-cd06-4fd9-a7c9-9bc0180a3fe9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:09:34.712600 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d33daf-ccb1-4fe9-8c77-0c2410725603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:09:34.713446 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d33daf-ccb1-4fe9-8c77-0c2410725603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:09:34.713731 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d33daf-ccb1-4fe9-8c77-0c2410725603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:09:34.714005 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d33daf-ccb1-4fe9-8c77-0c2410725603 admin admin] PATH_INFO: `/v3/users/64c8111203d34490a0db95a739f73e0b` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:09:34.715471 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d33daf-ccb1-4fe9-8c77-0c2410725603 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64c8111203d34490a0db95a739f73e0b)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:09:34.717443 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d33daf-ccb1-4fe9-8c77-0c2410725603 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:09:34.768441 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-82d33daf-ccb1-4fe9-8c77-0c2410725603 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64c8111203d34490a0db95a739f73e0b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:09:34.775482 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-82d33daf-ccb1-4fe9-8c77-0c2410725603 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64c8111203d34490a0db95a739f73e0b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:09:34.786770 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-82d33daf-ccb1-4fe9-8c77-0c2410725603 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64c8111203d34490a0db95a739f73e0b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:09:34.787467 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-82d33daf-ccb1-4fe9-8c77-0c2410725603 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64c8111203d34490a0db95a739f73e0b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:09:34.788117 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-82d33daf-ccb1-4fe9-8c77-0c2410725603 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64c8111203d34490a0db95a739f73e0b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:09:34.810853 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1469/2938] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:09:34 2022] DELETE /identity/v3/users/64c8111203d34490a0db95a739f73e0b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:09:34.993767 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102a1f2-9cb9-43bb-92b6-12ca638706ab service swift] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:09:35.005891 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102a1f2-9cb9-43bb-92b6-12ca638706ab service swift] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:09:35.076375 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987163a7-ccec-4944-a01e-3558a7b6ad34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:09:35.077245 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987163a7-ccec-4944-a01e-3558a7b6ad34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:09:35.077533 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987163a7-ccec-4944-a01e-3558a7b6ad34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:09:35.077939 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987163a7-ccec-4944-a01e-3558a7b6ad34 admin admin] PATH_INFO: `/v3/projects/6ca50c708922486a910456429af51e2e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:09:35.079043 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987163a7-ccec-4944-a01e-3558a7b6ad34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ca50c708922486a910456429af51e2e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:09:35.081141 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987163a7-ccec-4944-a01e-3558a7b6ad34 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:09:35.130161 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-987163a7-ccec-4944-a01e-3558a7b6ad34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ca50c708922486a910456429af51e2e', 'request_id': 'req-987163a7-ccec-4944-a01e-3558a7b6ad34'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:09:35.139588 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-987163a7-ccec-4944-a01e-3558a7b6ad34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ca50c708922486a910456429af51e2e', 'request_id': 'req-987163a7-ccec-4944-a01e-3558a7b6ad34'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:09:35.145770 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-987163a7-ccec-4944-a01e-3558a7b6ad34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ca50c708922486a910456429af51e2e', 'request_id': 'req-987163a7-ccec-4944-a01e-3558a7b6ad34'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:09:35.159072 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-987163a7-ccec-4944-a01e-3558a7b6ad34 admin admin] The token cache is being invalidate because project 6ca50c708922486a910456429af51e2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:09:35.159379 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-987163a7-ccec-4944-a01e-3558a7b6ad34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:09:35.160062 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-987163a7-ccec-4944-a01e-3558a7b6ad34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:09:35.161635 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1470/2939] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:09:34 2022] DELETE /identity/v3/projects/6ca50c708922486a910456429af51e2e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:09:35.168511 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76eb963-9a3d-4ae9-83d1-3386253b213c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:09:35.168721 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76eb963-9a3d-4ae9-83d1-3386253b213c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:09:35.169009 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76eb963-9a3d-4ae9-83d1-3386253b213c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:09:35.188767 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c76eb963-9a3d-4ae9-83d1-3386253b213c None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:09:35.189992 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c76eb963-9a3d-4ae9-83d1-3386253b213c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:09:35.253932 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1470/2940] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:09:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:09:35.262651 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0baf7fe-578f-43a8-904a-dc75a4973b5b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:09:35.263021 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0baf7fe-578f-43a8-904a-dc75a4973b5b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:09:35.263310 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0baf7fe-578f-43a8-904a-dc75a4973b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:09:35.285489 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e0baf7fe-578f-43a8-904a-dc75a4973b5b None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:09:35.286706 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e0baf7fe-578f-43a8-904a-dc75a4973b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:09:35.295045 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1471/2941] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:09:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:09:35.299334 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76eb963-9a3d-4ae9-83d1-3386253b213c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:09:35.310342 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76eb963-9a3d-4ae9-83d1-3386253b213c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:09:35.315160 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0887ac73-b649-4952-9458-390311ef65e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:09:35.315933 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0887ac73-b649-4952-9458-390311ef65e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:09:35.316170 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0887ac73-b649-4952-9458-390311ef65e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:09:35.316450 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0887ac73-b649-4952-9458-390311ef65e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:09:35.317187 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0887ac73-b649-4952-9458-390311ef65e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 20:09:35.317501 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0887ac73-b649-4952-9458-390311ef65e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:09:35.319124 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0887ac73-b649-4952-9458-390311ef65e5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:09:35.332499 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1471/2942] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Jul 25 20:09:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 20:09:35.336831 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0baf7fe-578f-43a8-904a-dc75a4973b5b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:09:35.347451 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0baf7fe-578f-43a8-904a-dc75a4973b5b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:09:35.352187 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b29887-b83e-4d21-afb8-eef48759bd90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:09:35.352852 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b29887-b83e-4d21-afb8-eef48759bd90 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:09:35.353139 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b29887-b83e-4d21-afb8-eef48759bd90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:09:35.353421 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b29887-b83e-4d21-afb8-eef48759bd90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:09:35.354127 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b29887-b83e-4d21-afb8-eef48759bd90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:09:35.356821 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b29887-b83e-4d21-afb8-eef48759bd90 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:09:35.391489 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1472/2943] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Jul 25 20:09:35 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 20:09:35.396420 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0887ac73-b649-4952-9458-390311ef65e5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:09:35.408178 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0887ac73-b649-4952-9458-390311ef65e5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:09:35.464333 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:09:35.476086 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3521d40b-7f97-4b73-99aa-94c67140beed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:09:35.476841 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521d40b-7f97-4b73-99aa-94c67140beed admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:09:35.477134 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521d40b-7f97-4b73-99aa-94c67140beed admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:09:35.477417 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521d40b-7f97-4b73-99aa-94c67140beed admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:09:35.478166 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3521d40b-7f97-4b73-99aa-94c67140beed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:09:35.480178 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3521d40b-7f97-4b73-99aa-94c67140beed admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:09:35.505169 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1472/2944] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Jul 25 20:09:35 2022] POST /identity/v3/users => generated 287 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 20:09:35.510113 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b29887-b83e-4d21-afb8-eef48759bd90 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:09:35.521630 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b29887-b83e-4d21-afb8-eef48759bd90 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:09:35.526339 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e7cb2-b488-4f69-ae6c-9378e30d090e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:09:35.527005 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e7cb2-b488-4f69-ae6c-9378e30d090e admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:09:35.528218 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e7cb2-b488-4f69-ae6c-9378e30d090e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:09:35.528807 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e7cb2-b488-4f69-ae6c-9378e30d090e admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:09:35.529297 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e7cb2-b488-4f69-ae6c-9378e30d090e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:09:35.530910 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e7cb2-b488-4f69-ae6c-9378e30d090e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:09:35.542100 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1473/2945] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 20:09:35.547142 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3521d40b-7f97-4b73-99aa-94c67140beed admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:09:35.557434 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3521d40b-7f97-4b73-99aa-94c67140beed admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:09:35.562422 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16c8bcf-aa86-4c5b-ab1c-0fcc9fcefe33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:09:35.563087 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c8bcf-aa86-4c5b-ab1c-0fcc9fcefe33 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:09:35.563372 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c8bcf-aa86-4c5b-ab1c-0fcc9fcefe33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:09:35.563683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c8bcf-aa86-4c5b-ab1c-0fcc9fcefe33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:09:35.564414 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16c8bcf-aa86-4c5b-ab1c-0fcc9fcefe33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:09:35.566011 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16c8bcf-aa86-4c5b-ab1c-0fcc9fcefe33 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:09:35.578011 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1473/2946] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Jul 25 20:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 20:09:35.582213 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e7cb2-b488-4f69-ae6c-9378e30d090e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:09:35.594346 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e7cb2-b488-4f69-ae6c-9378e30d090e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:09:35.599165 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f404c6e-73ab-40a7-a10f-e6c77a24608e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:09:35.600087 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f404c6e-73ab-40a7-a10f-e6c77a24608e admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:09:35.600377 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f404c6e-73ab-40a7-a10f-e6c77a24608e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:09:35.600682 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f404c6e-73ab-40a7-a10f-e6c77a24608e admin admin] PATH_INFO: `/v3/projects/141cd5a2d36d4540a63a6ecf818360d0/users/9eae725ddf7b46d4b85add3dc4a4416f/roles/c241bd9634e24bcbb1854c7621d19f81` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:09:35.618709 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f404c6e-73ab-40a7-a10f-e6c77a24608e admin admin] RBAC: Authorizing `identity:create_grant(project_id=141cd5a2d36d4540a63a6ecf818360d0, user_id=9eae725ddf7b46d4b85add3dc4a4416f, role_id=c241bd9634e24bcbb1854c7621d19f81)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:09:35.622689 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f404c6e-73ab-40a7-a10f-e6c77a24608e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:09:35.637239 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1474/2947] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Jul 25 20:09:35 2022] PUT /identity/v3/projects/141cd5a2d36d4540a63a6ecf818360d0/users/9eae725ddf7b46d4b85add3dc4a4416f/roles/c241bd9634e24bcbb1854c7621d19f81 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:09:35.648204 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dabc07f-43fe-495e-913d-4fb4c1d93629 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:09:35.648509 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dabc07f-43fe-495e-913d-4fb4c1d93629 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:09:35.648829 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dabc07f-43fe-495e-913d-4fb4c1d93629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:09:35.680222 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4dabc07f-43fe-495e-913d-4fb4c1d93629 None None] MFA Rules not processed for user `9eae725ddf7b46d4b85add3dc4a4416f`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:09:35.681463 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4dabc07f-43fe-495e-913d-4fb4c1d93629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:09:35.752280 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1474/2948] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:09:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:09:35.763538 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f404c6e-73ab-40a7-a10f-e6c77a24608e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:09:35.765405 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2f404c6e-73ab-40a7-a10f-e6c77a24608e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:09:35.781039 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:09:35.834635 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f404c6e-73ab-40a7-a10f-e6c77a24608e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:09:35.838330 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7e8dc-e54b-48d1-8aec-be4e4ac3831f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:09:35.838987 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7e8dc-e54b-48d1-8aec-be4e4ac3831f service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:09:35.839266 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7e8dc-e54b-48d1-8aec-be4e4ac3831f service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:09:35.839566 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7e8dc-e54b-48d1-8aec-be4e4ac3831f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:09:35.855979 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a7e8dc-e54b-48d1-8aec-be4e4ac3831f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:09:35.857488 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a7e8dc-e54b-48d1-8aec-be4e4ac3831f service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:09:35.867707 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1475/2949] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:09:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:09:45.749023 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=42,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:09:45.862789 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=20 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:09:46.118870 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dabc07f-43fe-495e-913d-4fb4c1d93629 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:09:46.197275 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dabc07f-43fe-495e-913d-4fb4c1d93629 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:09:46.204160 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d3d9c-d8f9-47cd-a5c8-b6b1de9002ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:09:46.205740 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d3d9c-d8f9-47cd-a5c8-b6b1de9002ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:09:46.206233 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d3d9c-d8f9-47cd-a5c8-b6b1de9002ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:09:46.206736 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d3d9c-d8f9-47cd-a5c8-b6b1de9002ef admin admin] PATH_INFO: `/v3/users/9eae725ddf7b46d4b85add3dc4a4416f` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:09:46.209045 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d3d9c-d8f9-47cd-a5c8-b6b1de9002ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eae725ddf7b46d4b85add3dc4a4416f)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:09:46.212990 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d3d9c-d8f9-47cd-a5c8-b6b1de9002ef admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:09:46.280212 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a04d3d9c-d8f9-47cd-a5c8-b6b1de9002ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eae725ddf7b46d4b85add3dc4a4416f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:09:46.288479 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a04d3d9c-d8f9-47cd-a5c8-b6b1de9002ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eae725ddf7b46d4b85add3dc4a4416f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:09:46.297576 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a04d3d9c-d8f9-47cd-a5c8-b6b1de9002ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eae725ddf7b46d4b85add3dc4a4416f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:09:46.298371 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a04d3d9c-d8f9-47cd-a5c8-b6b1de9002ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eae725ddf7b46d4b85add3dc4a4416f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:09:46.299085 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a04d3d9c-d8f9-47cd-a5c8-b6b1de9002ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eae725ddf7b46d4b85add3dc4a4416f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:09:46.320212 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1475/2950] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:09:46 2022] DELETE /identity/v3/users/9eae725ddf7b46d4b85add3dc4a4416f => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:09:46.326265 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bbe7e-ec3a-460c-b039-c6886039cf71 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:09:46.326704 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bbe7e-ec3a-460c-b039-c6886039cf71 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:09:46.326889 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bbe7e-ec3a-460c-b039-c6886039cf71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:09:46.348463 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6e1bbe7e-ec3a-460c-b039-c6886039cf71 None None] MFA Rules not processed for user `35d700f1abf943789efbf8a08bd9b587`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:09:46.349620 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6e1bbe7e-ec3a-460c-b039-c6886039cf71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:09:46.434481 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1476/2951] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:09:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:09:46.445387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d3d9c-d8f9-47cd-a5c8-b6b1de9002ef admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:09:46.446920 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a04d3d9c-d8f9-47cd-a5c8-b6b1de9002ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:09:46.521573 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d3d9c-d8f9-47cd-a5c8-b6b1de9002ef admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:09:46.525430 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea86ae-787b-4fdf-8c21-0ef8316c3a9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:09:46.526105 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea86ae-787b-4fdf-8c21-0ef8316c3a9f service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:09:46.526387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea86ae-787b-4fdf-8c21-0ef8316c3a9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:09:46.526663 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea86ae-787b-4fdf-8c21-0ef8316c3a9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:09:46.538060 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fea86ae-787b-4fdf-8c21-0ef8316c3a9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:09:46.539539 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fea86ae-787b-4fdf-8c21-0ef8316c3a9f service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:09:46.550726 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1476/2952] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:09:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 20:09:46.853622 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1bbe7e-ec3a-460c-b039-c6886039cf71 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:09:46.854874 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6e1bbe7e-ec3a-460c-b039-c6886039cf71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:09:46.875000 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1bbe7e-ec3a-460c-b039-c6886039cf71 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:09:46.883075 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed974974-0990-42b4-8fd4-e3db439e5918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:09:46.884365 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed974974-0990-42b4-8fd4-e3db439e5918 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:09:46.884807 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed974974-0990-42b4-8fd4-e3db439e5918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:09:46.885083 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed974974-0990-42b4-8fd4-e3db439e5918 admin admin] PATH_INFO: `/v3/projects/141cd5a2d36d4540a63a6ecf818360d0` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:09:46.887072 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed974974-0990-42b4-8fd4-e3db439e5918 admin admin] RBAC: Authorizing `identity:delete_project(project_id=141cd5a2d36d4540a63a6ecf818360d0)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:09:46.889730 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed974974-0990-42b4-8fd4-e3db439e5918 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:09:46.947969 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ed974974-0990-42b4-8fd4-e3db439e5918 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '141cd5a2d36d4540a63a6ecf818360d0', 'request_id': 'req-ed974974-0990-42b4-8fd4-e3db439e5918'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:09:46.957799 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ed974974-0990-42b4-8fd4-e3db439e5918 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '141cd5a2d36d4540a63a6ecf818360d0', 'request_id': 'req-ed974974-0990-42b4-8fd4-e3db439e5918'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:09:46.969178 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ed974974-0990-42b4-8fd4-e3db439e5918 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '141cd5a2d36d4540a63a6ecf818360d0', 'request_id': 'req-ed974974-0990-42b4-8fd4-e3db439e5918'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:09:46.982779 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ed974974-0990-42b4-8fd4-e3db439e5918 admin admin] The token cache is being invalidate because project 141cd5a2d36d4540a63a6ecf818360d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:09:46.983121 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ed974974-0990-42b4-8fd4-e3db439e5918 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:09:46.983839 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ed974974-0990-42b4-8fd4-e3db439e5918 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:09:46.985505 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1477/2953] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:09:46 2022] DELETE /identity/v3/projects/141cd5a2d36d4540a63a6ecf818360d0 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 20:09:56.545792 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:09:56.978529 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:10:37.473263 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab2891-34c8-4a9a-94a2-dab49a35f42b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:10:37.473831 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab2891-34c8-4a9a-94a2-dab49a35f42b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:10:37.473996 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab2891-34c8-4a9a-94a2-dab49a35f42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:10:37.495106 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a6ab2891-34c8-4a9a-94a2-dab49a35f42b None None] MFA Rules not processed for user `8143c33eabd74b1b9edc84e8ff155903`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:10:37.496744 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a6ab2891-34c8-4a9a-94a2-dab49a35f42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:10:37.581519 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1477/2954] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:10:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:10:47.571278 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=25 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:12:45.055393 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed974974-0990-42b4-8fd4-e3db439e5918 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:12:45.056869 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ed974974-0990-42b4-8fd4-e3db439e5918 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:12:45.137713 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed974974-0990-42b4-8fd4-e3db439e5918 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:12:45.141920 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8e1b7-05a9-466d-86a0-8db56a53bfbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:12:45.142565 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8e1b7-05a9-466d-86a0-8db56a53bfbe service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:12:45.142843 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8e1b7-05a9-466d-86a0-8db56a53bfbe service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:12:45.143121 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8e1b7-05a9-466d-86a0-8db56a53bfbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:12:45.144934 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-32b8e1b7-05a9-466d-86a0-8db56a53bfbe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:12:45.170751 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b8e1b7-05a9-466d-86a0-8db56a53bfbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:12:45.172510 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b8e1b7-05a9-466d-86a0-8db56a53bfbe service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:12:45.182541 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1478/2955] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:12:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:12:54.144794 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef989ae-db59-4587-9ebc-b3359302b17d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:12:54.145387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef989ae-db59-4587-9ebc-b3359302b17d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:12:54.145387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef989ae-db59-4587-9ebc-b3359302b17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:12:54.164453 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9ef989ae-db59-4587-9ebc-b3359302b17d None None] MFA Rules not processed for user `8143c33eabd74b1b9edc84e8ff155903`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:12:54.165681 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9ef989ae-db59-4587-9ebc-b3359302b17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:12:54.172875 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1478/2956] 10.222.0.18 () {58 vars in 958 bytes} [Mon Jul 25 20:12:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:12:54.212697 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8e1b7-05a9-466d-86a0-8db56a53bfbe service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:12:54.221807 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8e1b7-05a9-466d-86a0-8db56a53bfbe service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:12:54.225781 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61628a14-ecc9-4f7d-9586-12ffcb25dbb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aa720a35b9844848abcc42a4c4d6a2d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:12:54.226424 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61628a14-ecc9-4f7d-9586-12ffcb25dbb3 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:12:54.226709 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61628a14-ecc9-4f7d-9586-12ffcb25dbb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:12:54.226997 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61628a14-ecc9-4f7d-9586-12ffcb25dbb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:12:54.228914 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-61628a14-ecc9-4f7d-9586-12ffcb25dbb3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:12:54.304634 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61628a14-ecc9-4f7d-9586-12ffcb25dbb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:12:54.306099 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61628a14-ecc9-4f7d-9586-12ffcb25dbb3 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:12:54.317572 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1479/2957] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:12:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:12:55.100974 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c58f9-cabf-492f-9365-0c42b4281edc None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:12:55.101192 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c58f9-cabf-492f-9365-0c42b4281edc None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:12:55.101453 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c58f9-cabf-492f-9365-0c42b4281edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:12:55.155445 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ae5c58f9-cabf-492f-9365-0c42b4281edc None None] MFA Rules not processed for user `181c84715c094100b414cea3c4919eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:12:55.157566 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ae5c58f9-cabf-492f-9365-0c42b4281edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:12:55.235245 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1479/2958] 10.222.0.18 () {58 vars in 1062 bytes} [Mon Jul 25 20:12:55 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:12:55.453434 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61628a14-ecc9-4f7d-9586-12ffcb25dbb3 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:12:55.463841 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61628a14-ecc9-4f7d-9586-12ffcb25dbb3 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:12:55.467707 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc281213-cf40-43d0-ad8b-f16594f9643e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:12:55.468472 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc281213-cf40-43d0-ad8b-f16594f9643e service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:12:55.468894 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc281213-cf40-43d0-ad8b-f16594f9643e service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:12:55.469203 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc281213-cf40-43d0-ad8b-f16594f9643e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:12:55.471055 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-dc281213-cf40-43d0-ad8b-f16594f9643e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:12:55.486143 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc281213-cf40-43d0-ad8b-f16594f9643e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:12:55.487629 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc281213-cf40-43d0-ad8b-f16594f9643e service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:12:55.499408 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1480/2959] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:12:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:13:05.231582 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=40 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:13:05.494672 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=81 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:13:10.722249 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c58f9-cabf-492f-9365-0c42b4281edc None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:13:10.730387 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c58f9-cabf-492f-9365-0c42b4281edc None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:13:10.734250 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8771e25-69d4-4484-a848-65d8f801b914 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:13:10.734915 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8771e25-69d4-4484-a848-65d8f801b914 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:13:10.735205 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8771e25-69d4-4484-a848-65d8f801b914 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:13:10.735676 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8771e25-69d4-4484-a848-65d8f801b914 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:13:10.737651 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f8771e25-69d4-4484-a848-65d8f801b914 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:13:10.749309 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8771e25-69d4-4484-a848-65d8f801b914 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:13:10.750763 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8771e25-69d4-4484-a848-65d8f801b914 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:13:10.760496 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1480/2960] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:13:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:13:20.756602 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:13:55.993652 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc281213-cf40-43d0-ad8b-f16594f9643e service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:13:55.995144 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-dc281213-cf40-43d0-ad8b-f16594f9643e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:13:56.083547 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc281213-cf40-43d0-ad8b-f16594f9643e service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:13:56.089058 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3e76dd-699a-409d-8beb-7dc12b851853 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0422d6695bf14925bf0f993cb16e64ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:13:56.089694 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e76dd-699a-409d-8beb-7dc12b851853 service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:13:56.089976 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e76dd-699a-409d-8beb-7dc12b851853 service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:13:56.090291 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e76dd-699a-409d-8beb-7dc12b851853 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:13:56.092077 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7e3e76dd-699a-409d-8beb-7dc12b851853 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:13:56.104729 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3e76dd-699a-409d-8beb-7dc12b851853 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:13:56.106328 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3e76dd-699a-409d-8beb-7dc12b851853 service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:13:56.117511 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1481/2961] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:13:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:14:06.112393 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=39 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:15:01.115005 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8771e25-69d4-4484-a848-65d8f801b914 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:15:01.118171 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f8771e25-69d4-4484-a848-65d8f801b914 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:15:01.136001 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8771e25-69d4-4484-a848-65d8f801b914 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:15:01.141985 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d21f88b-37b3-45c6-9026-bc541acbc8a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:15:01.142736 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21f88b-37b3-45c6-9026-bc541acbc8a9 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:15:01.143095 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21f88b-37b3-45c6-9026-bc541acbc8a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:15:01.143448 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21f88b-37b3-45c6-9026-bc541acbc8a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:15:01.145472 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5d21f88b-37b3-45c6-9026-bc541acbc8a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:15:01.158985 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d21f88b-37b3-45c6-9026-bc541acbc8a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:15:01.160738 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d21f88b-37b3-45c6-9026-bc541acbc8a9 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:15:01.175681 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1481/2962] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:15:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:15:04.179687 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3e76dd-699a-409d-8beb-7dc12b851853 service placement] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:15:04.181423 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7e3e76dd-699a-409d-8beb-7dc12b851853 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:15:04.205072 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3e76dd-699a-409d-8beb-7dc12b851853 service placement] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:15:04.212208 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5ce17-a7c0-4425-ba42-95a0e5d87f68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '860e7d26ad364795bb25836aadd4ac7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:15:04.212879 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5ce17-a7c0-4425-ba42-95a0e5d87f68 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:15:04.213164 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5ce17-a7c0-4425-ba42-95a0e5d87f68 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:15:04.213442 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5ce17-a7c0-4425-ba42-95a0e5d87f68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:15:04.215707 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c0e5ce17-a7c0-4425-ba42-95a0e5d87f68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:15:04.229236 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e5ce17-a7c0-4425-ba42-95a0e5d87f68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:15:04.230839 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e5ce17-a7c0-4425-ba42-95a0e5d87f68 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:15:04.243111 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1482/2963] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:15:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:15:11.169482 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:15:14.237251 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=9 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:15:16.074784 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2b6d0-a15d-46c8-a580-05509872f0cd None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:15:16.075272 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2b6d0-a15d-46c8-a580-05509872f0cd None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:15:16.075568 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2b6d0-a15d-46c8-a580-05509872f0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:15:16.098614 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d4b2b6d0-a15d-46c8-a580-05509872f0cd None None] MFA Rules not processed for user `181c84715c094100b414cea3c4919eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:15:16.099925 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d4b2b6d0-a15d-46c8-a580-05509872f0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:15:16.107672 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1482/2964] 10.222.0.18 () {58 vars in 1062 bytes} [Mon Jul 25 20:15:16 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:15:16.113817 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5ce17-a7c0-4425-ba42-95a0e5d87f68 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:15:16.124117 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5ce17-a7c0-4425-ba42-95a0e5d87f68 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:15:16.127744 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad20a6d-7180-4d81-8631-ea7ea8ada270 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '181c84715c094100b414cea3c4919eb7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:15:16.128388 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad20a6d-7180-4d81-8631-ea7ea8ada270 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:15:16.128680 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad20a6d-7180-4d81-8631-ea7ea8ada270 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:15:16.128955 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad20a6d-7180-4d81-8631-ea7ea8ada270 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:15:16.130728 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1ad20a6d-7180-4d81-8631-ea7ea8ada270 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:15:16.199095 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad20a6d-7180-4d81-8631-ea7ea8ada270 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:15:16.200506 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad20a6d-7180-4d81-8631-ea7ea8ada270 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:15:16.210560 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1483/2965] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:15:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:15:19.310152 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b2b6d0-a15d-46c8-a580-05509872f0cd None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:15:19.322659 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b2b6d0-a15d-46c8-a580-05509872f0cd None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:15:19.327750 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb8e827-4316-4b25-8650-679e73f4b9ac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0422d6695bf14925bf0f993cb16e64ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:15:19.328629 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb8e827-4316-4b25-8650-679e73f4b9ac service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:15:19.328917 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb8e827-4316-4b25-8650-679e73f4b9ac service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:15:19.329196 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb8e827-4316-4b25-8650-679e73f4b9ac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:15:19.330969 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-acb8e827-4316-4b25-8650-679e73f4b9ac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:15:19.343704 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb8e827-4316-4b25-8650-679e73f4b9ac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:15:19.345305 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb8e827-4316-4b25-8650-679e73f4b9ac service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:15:19.361969 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1483/2966] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:15:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:15:22.652648 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad20a6d-7180-4d81-8631-ea7ea8ada270 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:15:22.654113 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1ad20a6d-7180-4d81-8631-ea7ea8ada270 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:15:22.667377 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad20a6d-7180-4d81-8631-ea7ea8ada270 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:15:22.671912 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749364f4-0cc8-4e76-918a-0ee3175dec0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:15:22.672793 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749364f4-0cc8-4e76-918a-0ee3175dec0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:15:22.673079 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749364f4-0cc8-4e76-918a-0ee3175dec0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:15:22.673357 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749364f4-0cc8-4e76-918a-0ee3175dec0b admin admin] PATH_INFO: `/v3/users/8143c33eabd74b1b9edc84e8ff155903` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:15:22.674555 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749364f4-0cc8-4e76-918a-0ee3175dec0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8143c33eabd74b1b9edc84e8ff155903)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:15:22.676599 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749364f4-0cc8-4e76-918a-0ee3175dec0b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:15:22.727496 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-749364f4-0cc8-4e76-918a-0ee3175dec0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8143c33eabd74b1b9edc84e8ff155903'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:15:22.734923 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-749364f4-0cc8-4e76-918a-0ee3175dec0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8143c33eabd74b1b9edc84e8ff155903'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:15:22.742114 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-749364f4-0cc8-4e76-918a-0ee3175dec0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8143c33eabd74b1b9edc84e8ff155903'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:15:22.742805 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-749364f4-0cc8-4e76-918a-0ee3175dec0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8143c33eabd74b1b9edc84e8ff155903'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:15:22.755020 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-749364f4-0cc8-4e76-918a-0ee3175dec0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8143c33eabd74b1b9edc84e8ff155903'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:15:22.761728 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1484/2967] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:15:22 2022] DELETE /identity/v3/users/8143c33eabd74b1b9edc84e8ff155903 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:15:22.939329 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb8e827-4316-4b25-8650-679e73f4b9ac service placement] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:15:22.954380 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb8e827-4316-4b25-8650-679e73f4b9ac service placement] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:15:23.010847 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56180177-fe43-4f97-bb81-05b513e5eafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:15:23.011742 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56180177-fe43-4f97-bb81-05b513e5eafc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:15:23.012026 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56180177-fe43-4f97-bb81-05b513e5eafc admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:15:23.012302 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56180177-fe43-4f97-bb81-05b513e5eafc admin admin] PATH_INFO: `/v3/projects/b6986405c7974cc8bdf5d0c470926c1f` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:15:23.013479 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56180177-fe43-4f97-bb81-05b513e5eafc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6986405c7974cc8bdf5d0c470926c1f)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:15:23.015385 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56180177-fe43-4f97-bb81-05b513e5eafc admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:15:23.061202 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-56180177-fe43-4f97-bb81-05b513e5eafc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6986405c7974cc8bdf5d0c470926c1f', 'request_id': 'req-56180177-fe43-4f97-bb81-05b513e5eafc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:15:23.067434 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-56180177-fe43-4f97-bb81-05b513e5eafc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6986405c7974cc8bdf5d0c470926c1f', 'request_id': 'req-56180177-fe43-4f97-bb81-05b513e5eafc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:15:23.074421 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-56180177-fe43-4f97-bb81-05b513e5eafc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6986405c7974cc8bdf5d0c470926c1f', 'request_id': 'req-56180177-fe43-4f97-bb81-05b513e5eafc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:15:23.087202 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-56180177-fe43-4f97-bb81-05b513e5eafc admin admin] The token cache is being invalidate because project b6986405c7974cc8bdf5d0c470926c1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:15:23.087490 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-56180177-fe43-4f97-bb81-05b513e5eafc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:15:23.088134 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-56180177-fe43-4f97-bb81-05b513e5eafc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:15:23.089673 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1484/2968] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:15:22 2022] DELETE /identity/v3/projects/b6986405c7974cc8bdf5d0c470926c1f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:15:23.093615 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749364f4-0cc8-4e76-918a-0ee3175dec0b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:15:23.095003 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-749364f4-0cc8-4e76-918a-0ee3175dec0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:15:23.107947 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749364f4-0cc8-4e76-918a-0ee3175dec0b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:15:23.166385 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452ec2e1-9aa3-499d-a742-4ad77a585db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:15:23.167173 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ec2e1-9aa3-499d-a742-4ad77a585db7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:15:23.167467 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ec2e1-9aa3-499d-a742-4ad77a585db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:15:23.167783 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ec2e1-9aa3-499d-a742-4ad77a585db7 admin admin] PATH_INFO: `/v3/users/b081268a6421470683561d5b6b743ac9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:15:23.169018 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452ec2e1-9aa3-499d-a742-4ad77a585db7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b081268a6421470683561d5b6b743ac9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:15:23.171428 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452ec2e1-9aa3-499d-a742-4ad77a585db7 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:15:23.177056 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:15:23.222972 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-452ec2e1-9aa3-499d-a742-4ad77a585db7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b081268a6421470683561d5b6b743ac9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:15:23.231878 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-452ec2e1-9aa3-499d-a742-4ad77a585db7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b081268a6421470683561d5b6b743ac9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:15:23.238875 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-452ec2e1-9aa3-499d-a742-4ad77a585db7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b081268a6421470683561d5b6b743ac9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:15:23.239557 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-452ec2e1-9aa3-499d-a742-4ad77a585db7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b081268a6421470683561d5b6b743ac9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:15:23.252223 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-452ec2e1-9aa3-499d-a742-4ad77a585db7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b081268a6421470683561d5b6b743ac9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:15:23.258432 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1485/2969] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Jul 25 20:15:23 2022] DELETE /identity/v3/users/b081268a6421470683561d5b6b743ac9 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:15:23.438353 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56180177-fe43-4f97-bb81-05b513e5eafc admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:15:23.439901 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-56180177-fe43-4f97-bb81-05b513e5eafc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:15:23.455619 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56180177-fe43-4f97-bb81-05b513e5eafc admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:15:23.520021 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2830bccb-73fd-41a7-93a9-ae0e47ed1a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d700f1abf943789efbf8a08bd9b587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9565eeec7ac7457abc5ee445dbf84c18', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:15:23.520887 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830bccb-73fd-41a7-93a9-ae0e47ed1a86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:15:23.521174 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830bccb-73fd-41a7-93a9-ae0e47ed1a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:15:23.521448 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830bccb-73fd-41a7-93a9-ae0e47ed1a86 admin admin] PATH_INFO: `/v3/projects/fd04bb0df5a44c98a8dd34d746ddb6d9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:15:23.522683 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2830bccb-73fd-41a7-93a9-ae0e47ed1a86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd04bb0df5a44c98a8dd34d746ddb6d9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:15:23.524638 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2830bccb-73fd-41a7-93a9-ae0e47ed1a86 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:15:23.535955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:15:23.581679 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2830bccb-73fd-41a7-93a9-ae0e47ed1a86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd04bb0df5a44c98a8dd34d746ddb6d9', 'request_id': 'req-2830bccb-73fd-41a7-93a9-ae0e47ed1a86'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:15:23.588848 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2830bccb-73fd-41a7-93a9-ae0e47ed1a86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd04bb0df5a44c98a8dd34d746ddb6d9', 'request_id': 'req-2830bccb-73fd-41a7-93a9-ae0e47ed1a86'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:15:23.594482 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2830bccb-73fd-41a7-93a9-ae0e47ed1a86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd04bb0df5a44c98a8dd34d746ddb6d9', 'request_id': 'req-2830bccb-73fd-41a7-93a9-ae0e47ed1a86'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:15:23.609476 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2830bccb-73fd-41a7-93a9-ae0e47ed1a86 admin admin] The token cache is being invalidate because project fd04bb0df5a44c98a8dd34d746ddb6d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 20:15:23.609798 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2830bccb-73fd-41a7-93a9-ae0e47ed1a86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:15:23.610460 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2830bccb-73fd-41a7-93a9-ae0e47ed1a86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 20:15:23.612085 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1485/2970] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Jul 25 20:15:23 2022] DELETE /identity/v3/projects/fd04bb0df5a44c98a8dd34d746ddb6d9 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 20:15:33.254598 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:15:33.604488 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=19,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:19:02.370749 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452ec2e1-9aa3-499d-a742-4ad77a585db7 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:19:02.372830 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-452ec2e1-9aa3-499d-a742-4ad77a585db7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:19:02.486295 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452ec2e1-9aa3-499d-a742-4ad77a585db7 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:19:02.551148 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1fb95d-02cc-4529-895c-36ba575b69e7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0422d6695bf14925bf0f993cb16e64ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:19:02.552105 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1fb95d-02cc-4529-895c-36ba575b69e7 service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:19:02.552404 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1fb95d-02cc-4529-895c-36ba575b69e7 service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:19:02.552685 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1fb95d-02cc-4529-895c-36ba575b69e7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:19:02.554625 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cd1fb95d-02cc-4529-895c-36ba575b69e7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:19:02.566798 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1fb95d-02cc-4529-895c-36ba575b69e7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:19:02.568424 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1fb95d-02cc-4529-895c-36ba575b69e7 service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:19:02.578451 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1486/2971] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:19:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:19:12.574114 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=67 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:21:05.352132 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b66e4-b4cb-483b-924a-696e450279de None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:21:05.352741 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b66e4-b4cb-483b-924a-696e450279de None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:21:05.352741 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b66e4-b4cb-483b-924a-696e450279de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:21:05.405623 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-463b66e4-b4cb-483b-924a-696e450279de None None] MFA Rules not processed for user `0422d6695bf14925bf0f993cb16e64ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 20:21:05.406884 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-463b66e4-b4cb-483b-924a-696e450279de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:21:05.416465 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1486/2972] 10.222.0.119 () {58 vars in 1016 bytes} [Mon Jul 25 20:21:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 20:21:05.430972 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1fb95d-02cc-4529-895c-36ba575b69e7 service placement] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 20:21:05.432440 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cd1fb95d-02cc-4529-895c-36ba575b69e7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 20:21:05.446007 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1fb95d-02cc-4529-895c-36ba575b69e7 service placement] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 20:21:05.451089 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ba8f7-e2fe-4883-96ff-4e38b812f1c8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0422d6695bf14925bf0f993cb16e64ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '191069caccd04233b012585e29cfa18b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 20:21:05.451771 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ba8f7-e2fe-4883-96ff-4e38b812f1c8 service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 20:21:05.452058 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ba8f7-e2fe-4883-96ff-4e38b812f1c8 service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 20:21:05.452336 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ba8f7-e2fe-4883-96ff-4e38b812f1c8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 20:21:05.462215 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ba8f7-e2fe-4883-96ff-4e38b812f1c8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 20:21:05.463842 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ba8f7-e2fe-4883-96ff-4e38b812f1c8 service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 20:21:05.475166 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1487/2973] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Jul 25 20:21:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 20:21:15.402955 dv-cinder-850939-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=13 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 20:21:15.469876 dv-cinder-850939-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}